DiligenceTasks

Description

DiligenceTasks is an agentic financial-diligence training environment. Each task poses an open-ended diligence question about a US public company — interpret a real disclosure pattern (funding/deposit repricing, capital adequacy, reserve sufficiency, cash-flow quality, off-balance-sheet exposure, segment/concentration, fair-value marks, liquidity/maturity, margin trajectory), take a directional credit/equity view, and surface the assumptions and missing data an analyst would flag. Tasks are generated from real SEC EDGAR 10-K/10-Q disclosures and graded against a weighted expert-style rubric by a gpt-5-mini judge.

Each query describes the disclosure pattern qualitatively but hides the exact figures, so the agent must recover them from the company's filings via the SEC EDGAR tools — a research-heavy signal ideal for RL. A held-out set of evaluation issuers is excluded from the training corpus.

The agent is given SEC EDGAR tools and web search, plus a shell/Python/file sandbox for exact calculation.

Capabilities

• Interpreting complex financial disclosures and taking a directional credit/equity view
• Primary-source diligence over SEC filings (10-K/10-Q), sections, and XBRL financial statements
• Numerical reasoning to tight tolerances (recovering exact reported figures with correct units and periods)
• Surfacing material assumptions and risks, and identifying the most-uncertain assumption and the most-important absent data
• Long-form synthesis with professional analyst judgement

Compute Requirements

Agents are given a sandbox with 2GB of RAM and 1 CPU (a Python data-science image) for calculations and scratch files. Web search and grading are API calls; SEC EDGAR access is a keyless public API.

License

MIT.

Tasks

A single train split with 456 tasks, each a single-issuer diligence question (265 single_period, from one recent 10-Q/10-K; 191 period_over_period, from two consecutive periodic filings for improved-vs-deteriorated comparisons). Issuers are drawn from NYSE/Nasdaq registrants; sector coverage follows the natural spread of the filing universe.

Reward Structure

Sparse, LLM-graded reward delivered once when the agent calls submit_answer. The answer is judged against the task's weighted rubric, organised into three sections — Factual Accuracy, Analytical Reasoning, Risk Awareness — with negative-weight criteria that penalize active errors (mischaracterising a trend, over-claiming certainty, ignoring an offsetting risk, omitting required assumptions). A gpt-5-mini judge labels each criterion MET/UNMET, and the reward is the normalized weighted sum of met criteria:

$R = \mathrm{clip}\!\left(\frac{\sum_{i \in \text{MET}} w_i}{\sum_{i:\, w_i > 0} w_i},\; 0,\; 1\right) \in [0, 1]$

A met negative-weight criterion subtracts from the numerator, so committing the errors the rubric anticipates actively lowers the score. As a calibration check, a research-grounded answer scores 1.0 on sampled tasks while an evasive "consult the filings and an advisor" answer scores 0.0 (and triggers the negative "no assumptions / generic data" criteria).

Data

Generated by the datagen/ pipeline (harvest → generate → verify) from real SEC EDGAR 10-K/10-Q filings:

• harvest pulls disclosure-rich windows (MD&A, market-risk / interest-rate sensitivity, credit-loss allowance, capital, fair value, liquidity, segment/concentration) plus a compact XBRL company-facts digest.
• generate (gpt-5.5) writes a figure-free diligence query and a weighted 3-section rubric; each Factual Accuracy criterion carries a verbatim source_quote from the filing.
• verify quote-grounds every factual criterion against the filing text, drops tasks whose query leaks the figures, audits every criterion with gpt-5-mini (atomicity, entailment, that negatives are genuine errors), and assembles the weighted rubric schema.

Data files are hosted on the OpenReward platform (the rubrics stay server-side; tasks expose only a dataset_id).

Tools

The environment exposes 17 tools: web_search, fetch_url (Tavily); the SEC EDGAR suite sec_resolve_company, sec_filings, sec_filing_content, sec_filing_search, sec_financials (keyless); the CLI/sandbox toolset (bash, glob, grep, ls, read, write, edit, multi_edit, todo_write); and submit_answer.

Time Horizon

Multi-turn agentic diligence episodes — research across SEC filings and the web, calculate in the sandbox, then submit once. In test rollouts, a gpt-5.2 agent used roughly 8–24 tool calls per task (dominated by sec_filing_search / sec_filing_content, plus bash for calculations) before submitting.

Environment Difficulty

Open-ended and research-demanding: the query hides the figures, so a strong answer requires recovering exact reported numbers from the filings and forming a defensible directional view. Reward is dense enough for RL (partial credit across ~25–35 weighted criteria per task) while penalizing the characteristic errors via negative weights. In three gpt-5.2 rollouts across different companies and themes, rewards clustered around 0.80–0.88, with the model recovering most factual figures and leaving points on the harder analytical/risk judgement — headroom that widens for weaker models. We have not yet run a full model sweep.

Other Environment Requirements

Two secrets are required:

• openai_api_key — for the gpt-5-mini grading judge
• tavily_api_key — for web_search / fetch_url

The SEC EDGAR tools require no API key (SEC's fair-access policy only asks for a descriptive User-Agent, which the environment sets).

Safety

The agent performs read-only research on public financial disclosures and produces a text analysis; it takes no real-world actions and handles no funds. Standard web-access considerations apply (it fetches live third-party content). Answers are research artifacts graded against rubrics, not investment advice, and models trained on this environment should not be treated as licensed financial advisors.

Citations

@misc{diligencetasks2026,
  title        = {DiligenceTasks: A Synthetic Training Environment for Financial Diligence},
  author       = {General Reasoning Inc. Team},
  year         = {2026},
  publisher    = {OpenReward},
  url          = {https://openreward.ai/GeneralReasoning/DiligenceTasks},
  note         = {Tasks generated from public SEC EDGAR filings.}
}