fs-review

Review a full set of financial statements the way an auditor does — as one interlinked whole — and flag where the numbers don't tie out.

Each task shows a complete mini statement set: an income statement, a retained-earnings reconciliation, a balance sheet, and a cash-flow summary. Some sets are clean; others hide a single defect that only surfaces when you check the statements against each other, not line by line. The agent reads all of it and reports what's wrong through the submit_findings tool.

Defects

Every set is built to tie out first, then one of three cross-statement errors is planted:

• unbalanced_balance_sheet — total assets ≠ total liabilities + equity
• retained_earnings_mismatch — opening RE + net income − dividends ≠ closing RE
• cash_mismatch — cash on the balance sheet ≠ closing cash on the cash-flow statement

Since each set is internally consistent apart from the one planted flaw, the answer key is unambiguous.

Tasks and splits

8 synthetic statement sets defined in server.py, split evenly and kept balanced:

Small on purpose — enough to prototype and see reward spread. To grow it, add entries to TRAIN_STATEMENTS / TEST_STATEMENTS.

Tools

Reward

Set-overlap (F1) between the reported and planted defects. Recall rewards catching real issues; precision punishes inventing them, so "flag everything" doesn't win — which keeps the reward honest as a training signal. Reporting a clean set correctly scores 1.0.

Because tool arguments are validated against a schema, there's no output-format reward to worry about here.

Run it locally

pip install -r requirements.txt
python server.py          # serves on http://0.0.0.0:8080

Then point a client at it:

from openreward import OpenReward

client = OpenReward()
env = client.environments.get(name="fs-review", base_url="http://localhost:8080")
tasks = env.list_tasks(split="test")

with env.session(task=tasks[0]) as session:
    print(session.get_prompt())
    result = session.call_tool("submit_findings", {"findings": []})
    print(result.reward)