UltimateTicTacToe

Description

UltimateTicTacToe is an environment for evaluating agents on strategic gameplay in Ultimate Tic-Tac-Toe, a complex variant where winning three sub-boards in a row determines victory. This environment wraps the UltimateTicTacToe implementation from TextArena, a framework for text-based game environments.

Capabilities

• Multi-level strategic planning across 9 interconnected Tic-Tac-Toe boards
• Evaluating agent ability to balance immediate tactical wins with long-term board control
• Testing opponent modeling and adaptive gameplay
• Assessing spatial reasoning across nested game structures

Compute Requirements

UltimateTicTacToe does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (50 tasks) and test (50 tasks). Each split contains 50 tasks across each of 1 variants:

• UltimateTicTacToe-v0

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• place_mark(board, position): Place your mark on a specific sub-board at a specific position. board is 0-8 (which sub-board), position is 0-8 (position within that sub-board).

Time Horizon

UltimateTicTacToe is a multi-turn environment.

Environment Difficulty

Medium. Ultimate Tic-Tac-Toe requires strategic thinking beyond simple Tic-Tac-Toe, balancing immediate sub-board tactics with overall board strategy. The nested structure creates complex decision spaces.

Other Environment Requirements

This environment requires an OpenAI API key (passed via secrets) to power the LLM opponent.

Safety

Agents in UltimateTicTacToe interact only with a strategic board game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}