Bandit

Description

Bandit is an environment for evaluating agents on the classic multi-armed bandit problem. This environment wraps the Bandit implementation from TextArena, a framework for text-based game environments.

Capabilities

• Exploration vs exploitation decision-making
• Learning from reward distributions
• Statistical reasoning under uncertainty

Compute Requirements

Bandit does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (300 tasks) and test (300 tasks). Each split contains 50 tasks across each of 6 variants:

• Bandit-v0: Standard multi-armed bandit
• Bandit-v0-train: Training variant with guidance
• Bandit-v0-raw: Raw feedback without formatting
• Bandit-v0-hard: Harder difficulty with closer reward distributions
• Bandit-v0-hard-train: Hard variant with training guidance
• Bandit-v0-hard-raw: Hard variant with raw feedback

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• press_button(button): Press a colored button. Each button has a different hidden reward distribution. Try to find the most rewarding button.

Time Horizon

Bandit is a multi-turn environment.

Environment Difficulty

Easy to Medium. The standard variants are accessible for learning basic exploration strategies, while the hard variants require more sophisticated approaches to distinguish between similar reward distributions.

Other Environment Requirements

There are no further environment requirements; Bandit works out of the box without any secrets or API keys.

Safety

Agents in Bandit interact only with a statistical decision-making game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}