Geoguessr

Description

Geoguessr is an environment for evaluating visual geographic reasoning. Given a street-view-style image, the agent must identify the country where the image was taken. An LLM grader evaluates whether the agent's guess matches the correct country, accounting for synonyms and naming variations.

Capabilities

• Visual geographic reasoning
• Country identification from street-view imagery
• Multi-modal image analysis

Compute Requirements

Agents are given a standard environment with no sandbox or file system access.

License

Images are sourced from Google Street View and are subject to Google's Terms of Service.

Tasks

There are two splits in this environment: train (100 tasks) and test (100 tasks), for a total of 200 tasks.

Each task presents a street-view image and the agent must guess the country where the image was taken.

Reward Structure

This is a single-turn environment. The agent submits a country guess via the guess tool. An LLM grader (gpt-4.1) evaluates whether the guess matches the correct country, accepting synonyms and naming variations (e.g., "USA" for "United States", "Korea" for "South Korea"). The reward is binary: 1.0 if correct, 0.0 if incorrect.

Data

Data consists of Parquet files (train.parquet, test.parquet) containing street-view images stored as binary JPEG data alongside country labels. Each file contains 100 rows. Data is stored on the OpenReward platform.

Tools

There is a single tool in this environment:

• guess — Submit a country name as the agent's guess. The tool grades the answer using an LLM and returns whether the guess was correct, an explanation, and the correct answer.

Time Horizon

Single-turn. The agent views the image and submits one country guess.

Environment Difficulty

Tasks require visual analysis of geographic clues such as road signs, vegetation, architecture, and driving side to identify countries. The environment tests multi-modal reasoning capabilities, as the agent must extract meaningful geographic signals from a single image.

Other Environment Requirements

OpenAI API key required for LLM-based grading. Pass via secrets={"openai_api_key": "..."} when creating the environment session.

Safety

Agents in Geoguessr identify countries from street-view images in a standard environment. The environment does not present direct safety risks.

Citation

@dataset{GeoguessrEnv,
  author    = {General Reasoning Inc. Team},
  title     = {GeoguessrEnv},
  year      = {2025},
  publisher = {OpenReward},
  url       = {https://openreward.ai/GeneralReasoning/Geoguessr}
}