TMax15K — OpenReward env

A dataset-driven terminal-agent env serving a curated subset of AI2's allenai/TMax-15K. Each task is a self-contained per-task Docker image (built from the task's container.def, fixtures baked in) plus a programmatic pytest verifier. The agent drives a bash shell across many turns and finishes by replying with a plain-text message; the hidden @terminal grader then runs the verifier against the container state.

This is the env-endless-terminals shape (per-task image + bash + pytest verifier), not a create-swe-env marathon. See ~/.claude/.../memory/tmax15k-env.md for the full investigation trail.

Reward signal

The reward is the task's own verifier (tests/test.sh → /logs/verifier/reward.txt), used outcome-only. No gold solution is needed to grade — AI2 ships none for the RL tasks by design (paper §3.1). Most verifiers recompute the expected result from the inputs or hardcode it inside test_final_state.py; a minority compare against a baked truth file.

De-leak (root-safe grading)

The platform runs the agent shell as root, so a ground-truth file cannot be hidden inside the agent's own container, and the paper documents real reward-hacking
(agents overwrote the checker / faked binaries). Therefore:

• setup() removes meta.truth_paths (baked answer/oracle files) from the agent's container before it acts. The %post/setup script is already gone (removed at image build, /tmp scrubbed), so the agent can't regenerate the answer for free.
• When the agent finishes with a plain-text reply, the hidden @terminal grader re-supplies the stripped ground truth (server-held copy the agent never saw), uploads the verifier fresh (overwriting any agent edit of /tests), runs it in the agent's container, and reads the reward from the verifier exit code observed by the env-server (not from any file the agent could fake).

meta.json per task: {truth_paths, output_paths, verifier_timeout}, derived in curation (read-by-test ∩ created-by-setup ∩ answer-named → truth; read-by-test ∖ created-by-setup → output) and confirmed by validate_tasks.py.

Pipeline (scripts/)

# 1. curate a stratified subset from the authoritative tasks.zip
python scripts/build_from_tmax.py --zip allenai_tasks.zip --n 50 --out ./tasks --clean

# 2. build per-task images (local; add --push to publish + record digests)
python scripts/build_images.py --push

# 3. validate: no-op must NOT pass; a known-good solution MUST pass (drops failures)
python scripts/validate_tasks.py --oraclepool oraclepool.parquet --prune

# 4. index the survivors
python scripts/index_tasks.py

Run locally

uv run python server.py        # serves http://0.0.0.0:8080

Then drive it with an OpenReward client (OpenReward(base_url="http://0.0.0.0:8080")): open a session, bash around, then reply with an ordinary message to finish (the harness will call session.call_terminal_tool() to grade). Empty work ⇒ reward 0; a correct solution ⇒ reward 1.

Status / scope

• v1: ~50 tasks, stratified across all 9 domains, file/exec verifiers only.
• Deferred to v2: multi_protocol/compose-stack/service tasks, tasks with
  non-enumerable output sets, the full 14,601 corpus.