FrontierFinance

Description

FrontierFinance is an agentic financial-research benchmark from Samaya AI. Each task is an expert-crafted investor query — asked as of a specific date — which the agent answers by researching the web (SEC filings, earnings call transcripts, company press releases, market data) and submitting a long-form answer. The answer is graded checklist-style against expert-authored rubrics.

Capabilities

• Financial research across the investor workflow: financial data/modeling, sector/industry/macro analysis, earnings/events, company research, coverage/catalyst monitoring, and screening/discovery
• Exhaustive retrieval (temporal, cross-entity, thematic) from primary sources
• Numerical reasoning over financial figures with correct units and periods
• Temporal anchoring: answering relative to the query's as-of date
• Long-form synthesis with professional investor judgement

Compute Requirements

This environment does not require a sandbox; compute requirements are minimal. Grading and web search are API calls.

License

CC-BY-4.0, matching the source dataset.

Tasks

A single train split with 220 tasks, one per benchmark query. Each task exposes only the query and its as-of date to the agent; the rubrics stay server-side. Queries span six use cases: financial data/modeling (70), sector/industry/macro (38), earnings/events (36), company research (32), coverage/catalyst monitoring (27), and screening/discovery (17).

Reward Structure

Sparse, LLM-graded reward delivered once when the agent calls submit_answer. The answer is judged against the query's expert-authored rubrics — 11,543 across the benchmark, ranging from 3 to 475 per query (mean ≈ 52) — each an atomic pass/fail criterion. The reward is the query's rubric qualification rate:

$R = \frac{\text{rubrics satisfied}}{\text{total rubrics}} \in [0, 1]$

Grading reimplements the official FrontierFinance grader (judge prompt used verbatim; rubrics judged in batches of 30 per call), with one deviation: we use a single gpt-5-mini judge, whereas the official evaluation takes a majority vote over a three-judge panel. Must-have rubrics (7,487 of 11,543) do not weight the reward — as in the official metrics — but the must-have qualification rate and per-rubric verdicts with reasons are returned in the tool metadata.

Data

Sourced from the samaya-ai/FrontierFinance dataset on Hugging Face (frontier_finance_public.jsonl): 220 queries with expert-authored rubrics, each rubric annotated with must-have status, rubric type, and required data-source type. Data files are hosted on the OpenReward platform.

Note that queries are anchored to past dates (as-of dates in 2024–2025) while the agent searches the live web, so it may encounter information published after the query date; the prompt instructs the agent to answer as of the query date, and the judge anchors temporal interpretation to that date.

Tools

• web_search — web search (Tavily); returns titles, URLs, and snippets
• fetch_url — fetch readable page content (Tavily extract), paginated at ~10,000 characters per page
• submit_answer — submit the long-form answer for grading; ends the episode

Time Horizon

Multi-turn agentic research episodes ending in a single submission. In our test rollouts with gpt-5.2, episodes used between 7 and 26 tool calls depending on query breadth.

Environment Difficulty

Samaya AI describe FrontierFinance as the hardest open finance benchmark, with the best evaluated system achieving roughly 50% (see the announcement). In our two spot-check rollouts, gpt-5.2 with the tools above scored 0.22 and 0.44 on two low-rubric-count tasks.

Other Environment Requirements

Two secrets are required:

• openai_api_key — for the gpt-5-mini grading judge
• tavily_api_key — for web_search / fetch_url

Safety

The agent performs read-only web research on public financial information and produces a text answer; it takes no real-world actions and handles no funds. Standard web-access considerations apply (the agent fetches live third-party content). Answers are research artifacts graded against rubrics, not investment advice, and models trained on this environment should not be treated as licensed financial advisors.

Citations

@article{zhang2026frontierfinance,
  title   = {FrontierFinance: A Benchmark for Measuring Frontier Intelligence of Finance Agents},
  author  = {Zhang, Yuhao and Koyluoglu, O. Ozan and Venkatesh, Thejas and Diehl Martinez, Richard and Bhatia, Vishank and Alidoust, Arash and Paranjape, Ashwin},
  year    = {2026},
  url     = {https://samaya.ai/blog/frontier-finance}
}