OpenRewardPolyCryptoUpDown
Polymarket Realtime Crypto Binaries
OpenReward environment that lets an agent paper-trade Polymarket's 5-minute
and 15-minute crypto "Up or Down" binary markets in real time, across every
underlying Polymarket lists them for (BTC, ETH, SOL, XRP, DOGE, BNB, HYPE, and
any new ones — matched by slug, not a fixed asset list). The env server connects
to Polymarket's live REST + WebSocket APIs, simulates taker fills against
its own cached order books, and emits per-settled-market USD-PnL reward.
The agent never places real orders on Polymarket — execution is fully
simulated. The agent's sandbox includes a pre-installed Polymarket L2
helper so it can subscribe to live order books directly for low-latency
decision making. Polymarket's public L2 market data feed is unauthenticated,
so no API token is provisioned to the sandbox.
What the agent sees
Market operations are not tools — they're REST endpoints on the env server
(server.py) that the agent's sandbox code calls through the pre-installed
polymarket_env_sandbox.bridge client. The only model-visible tools are the
sandbox CLI toolset (bash/read/write/edit/…) and collect_reward; the agent
writes and runs code to trade rather than issuing trade tool calls.
The bridge (and the routes behind it) expose:
active_markets(kind)— current crypto 5m/15m markets (all underlyings) with token ids, expiries, mids.kindfilters by asset ('BTC', 'SOL', ...), duration ('5m'/'15m'), exact 'ASSET-DURATION', or 'all'.place_trade(side, market_id, quantity)— FAK taker order against the live book; returns fills, VWAP, fee, new position.quote(side, market_id, quantity)— dry-run pricing of the same order (VWAP, fee, budget impact) without executing or touching the rate limit/budget.positions()— open positions with marks and unrealized PnL.balance()— running realized + unrealized PnL since session start.settlement_history()— markets settled this session and their PnL.
Each Polymarket binary has two outcome tokens (Up and Down); they are listed
and traded as separate positions. There is no shorting — SELL only closes
existing positions. To bet the other way, BUY the opposite outcome's token.
Reward
Reward is realized USD PnL, booked whenever it happens: at settlement
(shares × (outcome_price − avg_cost)) and at every fill (a SELL's
(vwap − avg_cost) × qty, and any fee), so the sum of rewards always equals
balance().realized_pnl. Market ops are REST calls, not tools, so reward can't
ride on them; instead PnL realized between tool calls is attached to the next
toolset tool call's ToolOutput.reward (bash/read/…), with breakdowns in
metadata.settlements and metadata.trade_pnl. This makes reward delivery independent of
any harness knowing about collect_reward, which remains as an explicit
force-a-drain. Multiple intervening settlements combine into a single emission.
Difficulty
Two knobs in the session's task_spec:
fee_multiplier (Θ ∈ [0, 1]) — a taker fee charged per fill leg:
fee_leg = fee_multiplier × 0.07 × size × p × (1 − p)
fee_multiplier=0 disables fees; 1.0 matches Polymarket's full crypto-market
taker fee (0.07 × p × (1−p)).
duration_filter ("all" | "5m" | "15m") — restricts the tradeable
universe to one market duration. "all" admits both; "5m"/"15m" admit only
that duration. Enforced server-side: active_markets() shows only in-scope
markets, and place_trade()/quote() on an out-of-scope market are rejected with
OUT_OF_SCOPE.
Preset tasks (split default) are the duration × fee cross-product — nine tasks,
ordered so the both-durations (all) tasks come first and, within each duration,
full fees come first, i.e. task[0] is the widest, full-fee universe:
| # | task_id | duration_filter | fee_multiplier |
|---|---|---|---|
| 0 | all_full_fees | all | 1.0 |
| 1 | all_half_fees | all | 0.5 |
| 2 | all_zero_fees | all | 0.0 |
| 3 | 5m_full_fees | 5m | 1.0 |
| 4 | 5m_half_fees | 5m | 0.5 |
| 5 | 5m_zero_fees | 5m | 0.0 |
| 6 | 15m_full_fees | 15m | 1.0 |
| 7 | 15m_half_fees | 15m | 0.5 |
| 8 | 15m_zero_fees | 15m | 0.0 |
Settlement buffer
All sessions enforce a fixed 60 s no-trade buffer before each market's
end_time (settlement_buffer_s in task_spec, defaulting to 60). Trades that
would land inside the window are rejected with SETTLEMENT_BUFFER. Without it,
agents harvest near-certain outcomes off the laggy order book in the final
seconds — giving a ~100% win rate that fees barely dent. Forcing entry ≥60 s
before close leaves real residual price uncertainty, so the edge has to come
from forecasting rather than picking up already-decided markets.
Local development
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e ".[dev]"
.venv/bin/python -m pytest tests/ # 55 tests, no network
.venv/bin/python scripts/local_verify.py # smoke test against live Polymarket
.venv/bin/python server.py # run the OpenReward server
Layout
env.py PolymarketEnvironment subclass + per-session state
server.py `python server.py` entrypoint + market-op REST routes
specs.py Pydantic models (config, request/result, settlement records)
prompt.py System prompt presented at session start
tasks.py Preset tasks / splits
polymarket_env/
polymarket_client.py REST + WS wrapper over Polymarket public APIs
market_tracker.py Polls Gamma every 30s for active crypto 5m/15m up/down markets (all underlyings)
book_feed.py Maintains per-token OrderBook cache from WS deltas
book.py OrderBook + L2 diff application
execution_sim.py FAK ladder-walking matcher
fees.py Per-leg taker fee
positions.py Portfolio, Position, realized/unrealized PnL
settlement.py Per-session settlement handler (fan-out target)
reward_queue.py Peek-then-ack settlement queue
shared.py Process-global Polymarket singleton + resolution detector
tools.py Market-op methods (REST-exposed) + collect_reward tool + reward drain
sandbox/
Dockerfile Image baked with the L2 helper module
polymarket_env_sandbox/ Agent-side helpers: L2 read client, price feed, trade bridge
tests/ Unit tests + e2e env test (no network)
scripts/
local_verify.py Live-Polymarket smoke test (one BUY against a real book)
live_settlement_test.py Live-Polymarket settlement-loop check
Open items
SANDBOX_IMAGEinenv.pyis built and pushed to GHCR by.github/workflows/build-sandbox.yml(taggedsha-<gitsha>) and pinned to an immutable digest tag. Repin it to the newsha-…the workflow prints after each sandbox rebuild.scripts/local_verify.pyrequires outbound network togamma-api.polymarket.com,clob.polymarket.com, andws-subscriptions-clob.polymarket.com.- Settlement resolution relies on Polymarket's
outcomePricesreturning to["1","0"]or["0","1"]. UMA can lag marketendDateby minutes; the settlement loop honors a 30 s grace period and retries every 5 s.