PolyBench

OpenReward environment where the agent researches real Polymarket prediction markets using a backdated web toolset, advances a simulated clock as it gathers evidence, and takes positions that settle against the markets' real resolutions. The score is the log-growth of its realized bankroll.

Modeled on the marketmaker env's spirit: a deterministic task surface, an auto-finishing session, and a dense closed-form (non-LLM) grader.

Two flavours (shared mechanics)

PolyBench ships in two flavours that share all the mechanics (the position book, deferred settlement, web tools, sandbox, reward) — that common code lives in env_base.py (BasePolymarketEnv); each flavour only supplies its task surface + prompt.

• GeneralReasoning/PolyBench — forecaster (default, at the repo root). Each task is one market: research it, take a position, hold to resolution. Pure outcome prediction. (env.py, tasks.py, prompt.md.)
• GeneralReasoning/PolyBench-Trader — trader (in trader/). Each task is a multi-market slate with a difficulty ladder (task 0 = 1 market, up to the full split). Trade several markets at once over the shared clock. (trader/env.py, trader/tasks.py, trader/prompt.md.)

Both are buy-only (open positions; no early selling) and use deferred settlement + realized-PnL reward.

The loop

Two mechanics keep the simulation honest:

• Web horizon: every web call and price lookup is bounded to a simulated "today" the agent can only move forward. The env exposes a live web_as_of the web toolset reads on every call, so future content is never visible.
• Deferred settlement: place_bet opens a position and reveals nothing — no outcome, no PnL. A position is realized only when advance_time moves the clock past its resolution date and it settles against the real outcome. So one market's result can't be learned (and used on a correlated, still-unresolved market) the instant you bet it.

Fills (liquidity cap)

A bet fills at the day's 00:00 UTC quote plus the taker fee, with no price impact, but only as much as the real market could absorb:

• Each history point dated d stores volume, the USD that traded in UTC day d−1 (the 24h before the d quote), and liquidity, the part of it traded at YES prices within ±5¢ of the d quote. Both use only trades before the quote, so neither looks ahead.
• On sim day d the env opens at most fill_cap_alpha × liquidity per market (fill_cap_alpha = 0.25 in the task spec, next to taker_fee_bps). The cap is shared by every bet on that market that day, YES and NO together, and resets the next day.
• A larger stake is partly filled; the unfilled part stays in cash. If less than min_stake can be filled (including no liquidity), nothing is opened and the session continues (finished=False). The output reports requested, filled and unfilled amounts and the capacity left today.
• A day without a history point (the price is carried forward over a gap) has no capacity; an earlier point's liquidity is never granted again.
• The forecaster leaves out a market when, in some tier, no day from the tier's start until trading ends can fill min_stake: every bet on it would be rejected.
• list_markets shows each market's fillable_today; price_history shows each day's volume. The reward formula is unchanged.

Layout

Scope & extensibility

A task is a slate of markets traded over many simulated days on one shared clock:

• Multi-market, multi-day, held positions: the agent may open positions on any tradeable market on any day. advance_time(days=N) (1–30 per call, repeatable) moves the shared clock forward and settles any position whose market has resolved. The clock stops at the last resolution date and the session auto-finishes when every market has resolved — there's no finish tool, so the agent can't bail early. Difficulty is the number of markets, exposed as a ladder of tasks (task 0 = 1 market, up to the full split).
• prompt.md is what the env renders — it never names a specific market; the agent calls list_markets.
• Real data: markets are real Polymarket markets in data/markets/<id>.json, built by infra/build_dataset.py from the Gamma API (question/category/resolved outcome) + CLOB (/prices-history, real daily YES prices) + the Data API taker trade tape (per-day volume/liquidity, infra/trade_tape.py), filtered to binary markets resolving inside the corpus window and genuinely uncertain at the window start. Markets resolving from June 2026 on were also selected for liquidity (the sum of their daily fill caps over their tradeable days is at least the $10k bankroll). Across the whole dataset each Gamma event keeps at most two legs, the most traded ones (infra/cap_events.py); every market records its event_id. Re-run the builder to refresh/extend; python infra/trade_tape.py re-derives the per-day fields of the shipped markets from their stored condition_id. polymarket/data.py (list_market_ids, load_market) is the only read path; POLY_DATA_DIR overrides the dir.
• Synthetic data for tests: polymarket/synthetic.py writes deterministic markets to data/markets_synthetic/; the test suite points POLY_DATA_DIR there (via tests/conftest.py) so it never needs the network or the real data.

Quickstart (local dev)

Python 3.11+.

python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt           # or: pip install -e <path-to>/or/python-sdk

# Drive the full tool surface in-process (no HTTP, no web backend)
python test_agent.py
python test_agent.py --split test --advance 14 --side NO --stake 300

# Tests (no network)
python -m pytest tests/ -q

Web toolset

web_search / web_fetch come from the SDK's openreward.toolsets.BackSearchToolset, against the live GR backdated corpus at search.openreward.ai. Every call's cutoff is read live off env.web_as_of, so the agent's web horizon tracks the simulated clock — no post-resolution leakage. The markets' dates sit inside Dec 2025 – Sep 2026 (CORPUS_START/CORPUS_END in infra/build_dataset.py), which the backdated corpus covers, so research as-of a simulated day returns real hits.

It's live only when the server process has OPENREWARD_API_KEY set (sent as the x-api-key header — your OpenReward key works). Relevant env vars:

Without the key the env still runs — only web_search/web_fetch are inert; list_markets, price_history, advance_time, and place_bet work offline.

Reward

Log-growth of the realized bankroll, scaled to ~percent units. It moves only when a position settles at its resolution date — open positions are marked-to-market for display but don't affect the score (intra-life price swings are noise, not a forecast result):

A lost bankroll scores the floor, −1000.

Safety

• A market's outcome is revealed only once the clock advances past its resolution date (deferred settlement); price history and web research are bounded to the current simulated day, and opening a position reveals nothing.
• Reward is deterministic, computed from a closed-form non-LLM grader.

License

MIT.