Counter

Description

Counter is a simple environment for evaluating an agent's ability to reach a target number using increment and decrement operations. The agent is given a starting count and a goal, and must use the available tools to manipulate the count to match the goal before submitting.

Capabilities

• Basic arithmetic reasoning
• Multi-step tool use
• Goal-directed planning

Compute Requirements

Agents are given a standard environment with no sandbox or file system access.

Tasks

There are two splits in this environment:

• train: 100 tasks
• test: 20 tasks

Each task specifies a starting count and a goal, both integers in the range [-50, 50].

Reward Structure

This is a multi-turn environment. The agent uses increment and decrement tools to adjust the count, then calls submit to check if the goal is reached. Each tool call returns a shaped reward of 1 / (|goal - count| + 1), which increases as the agent gets closer to the goal and equals 1.0 when the count matches the goal exactly. The submit tool only ends the episode when the count equals the goal.

Data

Tasks are procedurally generated at module load time using a fixed random seed. No external data files are required.

Tools

Time Horizon

Multi-turn. The agent adjusts the count over one or more steps and submits when ready.

Environment Difficulty

Counter is a simple environment intended primarily for testing and development. Any model capable of basic arithmetic should achieve near-perfect accuracy.

Other Environment Requirements

There are no further environment requirements.

Safety

Agents in Counter perform basic arithmetic operations in a standard environment. The environment does not present direct safety risks.

Citation

@software{counter_openreward,
  title={Counter Environment for OpenReward},
  author={{General Reasoning}},
  year={2025},
  url={https://www.openreward.ai/GeneralReasoning/Counter}
}