RealLawyer
RealLawyer
Description
RealLawyer is an environment for evaluating agents on realistic legal professional workflows. Agents act as corporate lawyers managing email communications, handling document attachments, and responding to clients in a simulated law firm setting. Tasks involve reading emails, downloading and processing attachments (PDF, Excel, Word, PowerPoint), and composing professional responses.
Capabilities
- Multi-turn email management and communication
- Document processing with PDF, Excel, Word, and PowerPoint tools
- Legal professional workflow simulation
- File system operations and organization
- Meeting scheduling with Zoom integration
Compute Requirements
Agents are given a sandbox with 0.5 CPU and 0.5GB RAM, with document processing tools (PDF, Excel, Word, PowerPoint) pre-installed.
Tasks
There is one split in this environment (test) with two tasks:
-
fundraising-intro: The agent receives an introduction email about a startup client (Adamic Inc) seeking legal counsel for a seed fundraising round. The agent must respond professionally, provide legal guidance on co-founder equity and fundraising sequencing, schedule a meeting, and organize client files according to firm procedures. Contains 5 subtasks across 4 stages.
-
time-sensitive-share-issuance: The agent handles an urgent founder share issuance for Adamic Inc. The agent must respond to the initial request, answer questions about fees, regulatory filings, and tax elections (83(b)/s.431), review attachments, and organize share-issuance materials in the firm's file hierarchy. Contains 4 subtasks across 4 stages.
Reward Structure
This is a multi-turn environment with rubric-based grading. Rewards are calculated based on:
- Subtask completion triggered by specific tool calls
- LLM grading (gpt-5-mini) of response quality against rubrics
- Weighted rubric scores normalized to 0.0-1.0
Data
Task data includes email threads, document attachments, and character profiles. Data is stored on the OpenReward platform.
Tools
CLI Tools:
| Tool | Description |
|---|---|
bash | Execute bash commands |
read | Read file contents |
write | Write files |
edit | Edit files |
glob | Find files by pattern |
grep | Search file contents |
ls | List directory contents |
todo_write | Task tracking |
Inbox Tools:
| Tool | Description |
|---|---|
read_inbox | Display email threads in Gmail-like format |
read_message | Display full email thread conversation |
download_attachments | Download attachments to drive |
respond_message | Reply to an email thread |
send_meeting_invite | Send Zoom meeting invite |
Document Toolsets:
- PDF tools: create, read, search, merge PDFs
- Excel tools: create spreadsheets, charts, data operations
- Word tools: create documents, formatting
- PowerPoint tools: create presentations
Time Horizon
Multi-turn. Agents manage email threads, process documents, and compose responses across multiple interactions until the task is resolved.
Environment Difficulty
Tasks require professional-level communication skills, document handling, and understanding of legal workflow conventions. Agents must coordinate multiple actions and maintain context across email threads.
Other Environment Requirements
OpenAI API key required for LLM-based grading and character responses. Pass via secrets={"openai_api_key": "..."}.
Safety
Agents in RealLawyer interact with simulated legal scenarios in a sandboxed environment. All data is synthetic and anonymized. The environment does not present direct safety risks.
Citation
@dataset{GRRealLawyer,
author = {General Reasoning Inc. Team},
title = {RealLawyer},
year = {2026},
publisher = {OpenReward},
url = {https://openreward.ai/GeneralReasoning/RealLawyer}
}