OpenReward

AutogradC

GeneralReasoning

GeneralReasoning/AutogradC

API Endpoint
Leaderboard
Loading leaderboard...
README

AutogradC

A "harness" OpenReward environment: the agent re-implements the core of
HIPS/autograd (automatic differentiation) from scratch in C. The artifact
autogradc reads one JSON case (a tensor expression graph + inputs + a derivative
task) on stdin and prints the value / gradient / Jacobian / JVP / Hessian-vector
product on stdout. It is graded against a corpus whose answers were computed
offline by the real Python autograd.

  • Spec / IO contract: build/PROTOCOL.md
  • Design + reward + anti-cheat: DESIGN.md
  • Env class (server-side grading): autogradc.py
  • Producer (sandbox, bwrap): harness/ (Rust ag-runner)
  • Corpus oracle (offline, author-run): build/generate_corpus.py
    → commits build/corpus/; build/filter_corpus.py splits it into both images.
  • Reference solution (proves the reward): reference_solution/

Layout

autogradc.py env class: tools (run_tests/progress), server-side grade_case server.py OpenReward server entrypoint Dockerfile env-server image (holds the expected answers) sandbox.Dockerfile sandbox image (C toolchain + corpus inputs + ag-runner) harness/ Rust producer: runs the artifact per case under bubblewrap build/PROTOCOL.md the IO contract (single source of truth) build/generate_corpus.py OFFLINE oracle (needs autograd); writes build/corpus/ build/filter_corpus.py splits committed build/corpus into the image layout build/corpus/ COMMITTED generated corpus (inputs + expected.jsonl) reference_solution/ from-scratch C reference (Phase-4 validation) golden_tests.py no-Docker invariant tests (telescoping reward, anti-cheat)

Workflow

  1. Generate the committed corpus once (author-time):
    uv run --with autograd --with numpy python build/generate_corpus.py
  2. uv run python -m pytest golden_tests.py -v (no Docker)
  3. docker build -f sandbox.Dockerfile -t autogradc:dev . (needs build/corpus committed)
  4. Reference-solution check: build reference_solution/ into a workspace, grade
    against the corpus → ~full; an empty stub → ~0.
  5. Publish: connect the repo to OpenReward (server image) + push to main (the
    workflow builds + pushes the sandbox image to GHCR); pin SANDBOX_IMAGE to the
    printed :sha-<gitsha>.
GeneralReasoning/AutogradC | OpenReward