FifteenPuzzle

Description

FifteenPuzzle is an ORS environment for evaluating agents on the classic sliding tile puzzle (also known as the 15-puzzle). This environment wraps the FifteenPuzzle implementation from TextArena, a framework for text-based game environments.

Capabilities

• Spatial reasoning and planning
• Heuristic search and state space exploration
• Sequential decision-making to reach goal states

Compute Requirements

FifteenPuzzle does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (150 tasks) and test (150 tasks). Each split contains 50 tasks across each of 3 variants:

• FifteenPuzzle-v0: Standard 15-puzzle with formatted feedback
• FifteenPuzzle-v0-train: Training variant with guidance
• FifteenPuzzle-v0-raw: Raw feedback without formatting

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• move(direction): Slide a tile by moving the blank space in the given direction (up, down, left, right).

Time Horizon

FifteenPuzzle is a multi-turn environment.

Environment Difficulty

Medium to Hard. The puzzle requires planning ahead and understanding of solvability constraints, with optimal solutions often requiring long sequences of moves.

Other Environment Requirements

There are no further environment requirements; FifteenPuzzle works out of the box without any secrets or API keys.

Safety

Agents in FifteenPuzzle interact only with a sliding tile puzzle game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}