Fixtures

CCPA has five distinct fixture corpora, each measuring a different thing.

1. fixtures/canonical/ — the meter

  • 30 fixtures, every required-row of apr-code-parity-v1.yaml exercised at least once.
  • AUTHORED teacher/student trace pairs.
  • MUST score ≥ threshold in ccpa corpus. Per-PR CI hard-blocker via FALSIFY-CCPA-007.
  • Aggregate parity = 1.0000 at canonical corpus (M150, fixtures/canonical/measured-parity.json).

2. fixtures/regression/ — bidirectional sensitivity proof

  • Fixtures with deliberate drift — teacher and student diverge in known ways.
  • MUST FAIL ccpa corpus. If a regression fixture passes, the differ has lost sensitivity to that drift class.
  • Catches "the meter agrees on everything" bugs (M9 introduced this corpus).

3. fixtures/project-scale/ — Phase 5 Arena corpus

  • 5 real GitHub-issue Rust fixtures with full cwd-tree/, prompt.txt, oracle.
  • Each fixture is a real Rust bug or feature request that an agent must solve in a multi-turn session.
  • M234 finding: claude 1/5, apr code 0/5. Direction agrees with static verdict; magnitudes diverge.

4. fixtures/calibration-and-scale/ — synthetic-deterministic project-scale

  • 15 hand-authored Rust bug fixtures.
  • Deterministic seed; reproducible from clean clone.
  • Bridges the static path (controlled) and project-scale Arena (real-world) via a controlled Arena-style measurement.

5. fixtures/under-contract/ — Phase 6 corpus

  • 20 fixtures across 4 classes: leetcode, oo (OO patterns), transpile (format converters), unix (CLI utilities).
  • Each runs under the Phase 6 compound oracle: cargo test AND pmat comply check --strict.
  • The corpus that V1_004 dispatches against.

Fixture file layout

fixtures/canonical/0001-edit-readme/
├── meta.toml                       # fixture id, covers[], description
├── teacher.ccpa-trace.jsonl        # AUTHORED teacher action stream
└── student.ccpa-trace.jsonl        # AUTHORED student action stream

fixtures/under-contract/leetcode/01-two-sum/
├── prompt.txt                      # the task description shown to both agents
├── meta.toml                       # oracle_cmd, expected_pattern
└── cwd-tree/
    ├── Cargo.toml
    ├── src/lib.rs                  # the buggy code
    └── tests/...

Adding a fixture

mkdir fixtures/canonical/00XX-my-scenario

cat > fixtures/canonical/00XX-my-scenario/meta.toml <<EOF
[fixture]
id = "00XX-my-scenario"
covers = ["builtin-tools-rwegs"]
description = "What this fixture exercises and why."
EOF

# Author teacher.ccpa-trace.jsonl + student.ccpa-trace.jsonl

ccpa corpus fixtures/canonical/                            # MUST exit 0
ccpa coverage --apr-code-parity-yaml ... --oos-rows ...    # MUST exit 0
make tier3                                                 # full local gate sweep

Coverage gates fail if a fixture is added without a covers[] claim or if covers[] contains a row not in apr-code-parity-v1.yaml. The contract YAML drives fixture validation, not the other way around.