LogicPuzzle

Description

LogicPuzzle is an environment for evaluating agents on deductive reasoning and logic grid puzzle solving. This environment wraps the LogicPuzzle implementation from TextArena, a framework for text-based game environments.

Capabilities

• Deductive reasoning from clues
• Logic grid puzzle solving
• Tracking relationships between items
• Constraint satisfaction and inference

Compute Requirements

LogicPuzzle does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

The number of variants is dynamically determined from the TextArena registry (all variants with "LogicPuzzle" prefix). There are two splits: train and test. Each split contains 50 tasks per variant.

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• mark_cell(row_item, column_item, mark): Mark a cell in the logic grid.

Time Horizon

LogicPuzzle is a multi-turn environment.

Environment Difficulty

This environment presents moderate to challenging difficulty, requiring agents to systematically deduce relationships between items based on given clues and mark cells accordingly.

Other Environment Requirements

There are no further environment requirements; LogicPuzzle works out of the box without any secrets or API keys.

Safety

Agents in LogicPuzzle interact only with a logic grid puzzle and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}