Spreadsheet-RL

Description

Spreadsheet-RL is an environment for evaluating and training language model agents on realistic Excel workbook manipulation. Each task gives the agent a starting workbook and a natural-language instruction describing a desired end state (e.g. "add computed columns starting at column Q…", "populate the Velocity_Summary sheet…"). The agent edits the workbook in a sandbox using spreadsheet and CLI tools, then submits it for grading. The submitted workbook is the answer: it is recalculated and compared against a private oracle workbook over a task-specified cell region.

This is a reimplementation of the benchmark from Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning. The test split is SpreadsheetBench; the train split is the paper's filtered ExcelForum task collection.

Capabilities

• Reading, inspecting, and editing multi-sheet Excel workbooks
• Writing Excel formulas and static values into targeted cell ranges
• Structural edits (adding/removing sheets, clearing ranges) via native tools or Python
• Long-horizon multi-turn tool use with a final verifiable outcome

Compute Requirements

Agents are given a sandbox with 1 CPU and 2GB RAM, network access enabled, and a Python 3.12 data science image (ships openpyxl and pandas).

License

CC-BY-SA-4.0, matching the Spreadsheet-RL dataset license.

Tasks

There are 8,647 tasks across 2 splits:

Each task provides a starting workbook (input.xlsx), a natural-language instruction, and a private oracle workbook (target.xlsx) together with an answer position — the sheet(s) and cell range(s) over which the result is graded. Instructions span cell-level and sheet-level manipulations (formulas, aggregations, filtering, formatting, structural edits) drawn from real spreadsheet questions.

Reward Structure

This is a verifiable, non-LLM-graded reward environment. On submit, the agent's workbook is recalculated and compared cell-by-cell against the oracle within the answer-position region:

$\text{reward} = \frac{\text{cells matching the oracle}}{\text{cells in the answer region}}$

A cell matches under value-level rules: numeric comparison with tolerance, blank/None equivalence, and whitespace/case-normalized text. The reward is a fractional region cell-match score (dense signal, useful for RL), and the returned metadata includes a pass flag that is True only when every region cell matches (the paper's Pass@1 metric). An unreadable or corrupt submission scores 0.0.

Formulas are recalculated with LibreOffice (soffice) headless before comparison, following the SpreadsheetBench open-source evaluator. The original paper recalculates with native Microsoft Excel for maximum fidelity; a small number of Excel-only dynamic-array functions may recalculate imperfectly under LibreOffice.

Data

Task workbooks and parquet task tables are sourced from the Spreadsheet-RL dataset on Hugging Face and hosted on the OpenReward platform. Each task's starting workbook is uploaded into the sandbox for editing; the oracle workbook is held privately on the environment host and is only read during grading.

Tools

Agents use the OpenReward SDK's ExcelToolset and CLIToolset, plus a submit tool:

• Excel tools: excel_list_tabs_in_spreadsheet, excel_read_tab, excel_edit_spreadsheet (set a cell; values beginning with = are formulas), excel_add_content_text (write a 2D block), excel_delete_content_cell, excel_add_tab, excel_delete_tab, excel_create_spreadsheet, excel_read_csv, excel_create_chart.
• CLI tools: bash, read, write, edit, ls, glob, grep — e.g. to run Python with openpyxl/pandas for operations the native tools do not cover (filling a formula across a range, bulk transforms).
• submit: grade the current workbook against the oracle and finish the task.

Time Horizon

Spreadsheet-RL is a multi-turn environment. The agent inspects and edits the workbook over multiple tool calls, then calls submit once to be graded and finish.

Other Environment Requirements

There are no further environment requirements beyond the OpenReward platform; the environment works with the OpenReward endpoint without any external API keys.

Safety

Agents in Spreadsheet-RL only read and edit spreadsheet files within a sandboxed environment; no real-world systems are affected. The environment does not present direct safety risks.

Citations

@article{spreadsheetrl2026,
  title   = {Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning},
  author  = {Chi, Banghao and Xie, Yining and others},
  journal = {arXiv preprint arXiv:2605.22642},
  year    = {2026},
  url     = {https://arxiv.org/abs/2605.22642}
}

@inproceedings{ma2024spreadsheetbench,
  title     = {SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation},
  author    = {Ma, Zeyao and Zhang, Bohan and Zhang, Jing and Yu, Jifan and Zhang, Xiaokang and Zhang, Xiaohan and Luo, Sijia and Wang, Xi and Tang, Jie},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2024},
  url       = {https://github.com/RUCKBReasoning/SpreadsheetBench}
}