MemoryGame

Description

MemoryGame is an environment for evaluating agents on the classic memory/concentration card-matching game. This environment wraps the MemoryGame implementation from TextArena, a framework for text-based game environments.

Capabilities

• Memory retention and recall
• Pattern recognition and matching
• Sequential reasoning and state tracking
• Two-player competitive gameplay against an LLM opponent

Compute Requirements

MemoryGame does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (450 tasks) and test (450 tasks). Each split contains 50 tasks across each of 9 variants:

• MemoryGame-v0
• MemoryGame-v0-hard
• MemoryGame-v0-hard-raw
• MemoryGame-v0-hard-train
• MemoryGame-v0-medium
• MemoryGame-v0-medium-raw
• MemoryGame-v0-medium-train
• MemoryGame-v0-raw
• MemoryGame-v0-train

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• flip_cards(row1, col1, row2, col2): Flip two cards by specifying their positions (row1, col1) and (row2, col2). Cards are in a grid with rows and columns starting at 0.

Time Horizon

MemoryGame is a multi-turn environment.

Environment Difficulty

Easy to Moderate. Memory Game tests recall and pattern recognition, with difficulty scaling based on board size. The core mechanic is straightforward but requires tracking previously revealed cards.

Other Environment Requirements

This environment requires an OpenAI API key (passed via secrets) to power the LLM opponent.

Safety

Agents in MemoryGame interact only with a card-matching game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}