Cryptarithm

Description

Cryptarithm is an ORS environment for evaluating agents on cryptarithmetic puzzles, where letters must be mapped to unique digits to make an arithmetic equation valid. This environment wraps the Cryptarithm implementation from TextArena, a framework for text-based game environments.

Capabilities

• Logical reasoning and constraint satisfaction
• Systematic search and backtracking
• Mathematical problem solving

Compute Requirements

Cryptarithm does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (150 tasks) and test (150 tasks). Each split contains 50 tasks across each of 3 variants:

• Cryptarithm-v0: Standard cryptarithmetic puzzle
• Cryptarithm-v0-train: Training variant with guidance
• Cryptarithm-v0-raw: Raw feedback without formatting

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• assign_digit(letter, digit): Assign a digit (0-9) to a letter. Each letter must map to a unique digit to make the arithmetic equation valid.

Time Horizon

Cryptarithm is a multi-turn environment.

Environment Difficulty

Medium to Hard. The puzzles require systematic constraint satisfaction and logical reasoning to find the unique mapping of letters to digits.

Other Environment Requirements

There are no further environment requirements; Cryptarithm works out of the box without any secrets or API keys.

Safety

Agents in Cryptarithm interact only with a mathematical puzzle game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}