CvxpyRust

A long-horizon OpenReward environment: the agent re-implements the core of
cvxpy (a convex-optimization modeling library) from scratch in pure-std
Rust as cvxsolve — DCP curvature/sign analysis, the is_dcp verdict, and a
from-scratch convex solver — graded server-side against a cvxpy-generated corpus.
Published as GeneralReasoning/CvxpyRust.

See DESIGN.md for the full design and build/PROTOCOL.md for the artifact
invocation + grading contract.

Reward

Per-build delta of the build's score: the current build's score minus the
previous build's, so regressions are negative. Reward tracks the current
artifact, so the trajectory total telescopes to the score of the build the
agent finishes with. finished=True only when a single build passes every
case at once.

Each tier is worth its case count (857 in all). An answer that ignores the input
earns 0:

• classification questions (curvature and sign in tier1, half a case each;
  is_dcp in tier2) earn chance-corrected balanced accuracy,
  cases × (BA − 1/K)/(1 − 1/K) (Youden's J for is_dcp), unclipped. A missing
  or unusable answer counts as unknown / false, so a crash scores 0;
• solve cases a constant answer passes (infeasible/unbounded, and v* = 0) are
  credited in proportion to the pass rate on the tier's other cases.

run_tests

• timeout_s is clamped to 60–3600s; the producer's sandbox.run gets a deadline
  above it and the build its own 900s one (the SDK default is 300s).
• Every producer process runs under a 1.5 GiB address-space limit (ulimit -v), so
  a case that allocates without bound fails on its own. If the sandbox still runs
  out of memory (e.g. huge files under /tmp), run_tests returns "ran out of
  memory" feedback (reward 0, baseline untouched, no grade charged) and stops the
  leftover runner/jail processes and scratch files instead of raising.
• The graded corpus is NOT on the sandbox: the env-server image holds it and each
  grade stages only the questions into a random /tmp dir, deleted afterwards.
• Calls are serialized per session, so a concurrent build can't swap the binary
  under a running grade.
• The tool output is bounded (metadata included). The full build log and the full
  report go to /tmp/run_tests/{build.log,report.txt} in the sandbox (previous
  build: .prev), written in chunks and best-effort (a sabotaged /tmp can't
  error the rollout). Per-case pass/fail is deliberately NOT written: the answers
  are small categorical labels (see DESIGN.md).

Layout

Develop / test

# No-Docker invariant tests:
uv run python -m pytest golden_tests.py -v

# Local sandbox-image build (iterate / reference-solution check). On push to main
# .github/workflows/build-sandbox.yml builds + pushes :latest + :sha-<gitsha> to
# GHCR; then repin SANDBOX_IMAGE to the new :sha-<gitsha> (NOT :latest — the GHCR
# mirror serves :latest stale). Build + push by hand only as a fallback:
docker build --platform linux/amd64 -f sandbox.Dockerfile -t ghcr.io/generalreasoning/cvxpyrust:latest .
docker push ghcr.io/generalreasoning/cvxpyrust:latest

# Local server smoke test:
uv run python server.py            # in one shell
uv run python test_agent.py        # in another

Corpus regeneration (author-time only)

# needs cvxpy; produces the committed build/corpus/ (do not run in either image)
python build/generate_corpus.py --out build/corpus
# discrimination check against a built reference cvxsolve:
python build/local_grade.py --binary reference_solution/target/release/cvxsolve
python build/local_grade.py --baseline   # the ~25% always-constant floor to beat