GuessTheNumber

Description

GuessTheNumber is an ORS environment for evaluating agents on the classic number guessing game where agents receive feedback on whether their guess is too high or too low. This environment wraps the GuessTheNumber implementation from TextArena, a framework for text-based game environments.

Capabilities

• Binary search and systematic exploration
• Information-theoretic reasoning
• Adaptive strategy based on feedback

Compute Requirements

GuessTheNumber does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (300 tasks) and test (300 tasks). Each split contains 50 tasks across each of 6 variants:

• GuessTheNumber-v0: Standard number guessing with formatted feedback
• GuessTheNumber-v0-train: Training variant with guidance
• GuessTheNumber-v0-raw: Raw feedback without formatting
• GuessTheNumber-v0-hardcore: Harder difficulty with larger range
• GuessTheNumber-v0-hardcore-train: Hardcore with training guidance
• GuessTheNumber-v0-hardcore-raw: Hardcore with raw feedback

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• guess_number(number): Guess a number. You'll be told if it's too high, too low, or correct.

Time Horizon

GuessTheNumber is a multi-turn environment.

Environment Difficulty

Easy to Medium. The standard variants are straightforward and can be solved optimally with binary search, while the hardcore variants have larger search spaces requiring more guesses.

Other Environment Requirements

There are no further environment requirements; GuessTheNumber works out of the box without any secrets or API keys.

Safety

Agents in GuessTheNumber interact only with a number guessing game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}