Hangman

Description

Hangman is an ORS environment for evaluating agents on word guessing and deductive reasoning tasks. This environment wraps the Hangman implementation from TextArena, a framework for text-based game environments.

Capabilities

• Deductive reasoning with partial information
• Strategic letter selection and word guessing
• Managing limited attempts under uncertainty
• Pattern recognition in word structures

Compute Requirements

Hangman does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (300 tasks) and test (300 tasks). Each split contains 50 tasks across each of 6 variants:

• Hangman-v0
• Hangman-v0-train
• Hangman-v0-raw
• Hangman-v0-hardcore
• Hangman-v0-hardcore-train
• Hangman-v0-hardcore-raw

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given two tools:

• guess_letter(letter): Guess a single letter. If it's in the word, it will be revealed.
• guess_word(word): Guess the entire word at once. Correct guess wins; wrong guess costs an attempt.

Time Horizon

Hangman is a multi-turn environment.

Environment Difficulty

This environment ranges from moderate to challenging depending on the variant. The hardcore variants present increased difficulty with more complex word patterns and stricter attempt limits.

Other Environment Requirements

There are no further environment requirements; Hangman works out of the box without any secrets or API keys.

Safety

Agents in Hangman interact only with a word-guessing game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}