Crosswords

Description

Crosswords is an environment for evaluating cryptic crossword clue solving. It contains 660,613 clues from various puzzle sources including The Times, Guardian, and other publications. Agents must decode cryptic wordplay to determine the correct answer for each clue.

Capabilities

• Cryptic crossword clue interpretation
• Wordplay and linguistic reasoning
• Pattern recognition and anagram solving

Compute Requirements

Agents are given a standard environment with no sandbox or file system access.

License

ODbL.

Tasks

There are three splits in this environment:

• train: 528,490 tasks (80%)
• val: 66,061 tasks (10%)
• test: 66,062 tasks (10%)

Each task presents a cryptic crossword clue with optional metadata (puzzle name, date, clue number).

Reward Structure

This is a single-turn environment. The agent submits an answer via the submit_answer tool. Validation uses exact string matching with normalization (case-insensitive, punctuation and whitespace removed). Reward is binary: 1.0 if correct, 0.0 if incorrect.

Data

Data consists of a single Parquet file (crosswords.parquet, ~36 MB) sourced from HuggingFace jeggers/crosswords. Each row contains the clue text, answer, puzzle metadata, and optional definition field. Data is stored on the OpenReward platform.

Tools

Time Horizon

Single-turn. The agent reads the clue and submits one answer.

Environment Difficulty

Cryptic crossword solving requires sophisticated linguistic reasoning. The benchmark tests wordplay interpretation, anagram solving, and cryptic definition parsing.

Other Environment Requirements

There are no further environment requirements; Crosswords works out of the box with the OpenReward endpoint without any external API keys.

Safety

Agents in Crosswords solve cryptic crossword clues in a standard environment. The environment does not present direct safety risks.

Citation

@dataset{jeggers_crosswords,
  author    = {jeggers},
  title     = {Crosswords Dataset},
  year      = {2023},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/jeggers/crosswords}
}