Skip to content

test: end-to-end golden-output regression test - #17

Merged
bnemoz merged 1 commit into
mainfrom
tests/golden-output
Nov 17, 2026
Merged

bnemoz merged 1 commit into
mainfrom
tests/golden-output

Conversation

@bnemoz

@bnemoz bnemoz commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Adds an end-to-end regression test, so changes to PairPlex can't silently alter the reference output.

What

  • Fixture (pairplex/tests/data/golden/input/, 22 KB): two simulated wells with 37 barcodes and about 500 reads each, built from SimPlex output of public paired antibody data.
    • Each well stays under 1000 reads. PairPlex splits input into 1000-read chunks and collects them in completion order, so this keeps the read order into clustering deterministic.
    • The run gives 140 clusters (some failing the filters), 111 consensus sequences and 24 pairs.
  • Golden tables (expected/): metadata, consensus, unpaired and paired. They come from main at a8b6376 (v2-base plus fix: always pass an explicit mmseqs thread count to abstar.run #12 and fix(cli): make-fastq passes the wrong keyword argument #13, neither of which changes output), with abstar 0.9.0.
  • test_golden.py:
    • Reference behaviour must equal the golden tables. Reference means run(legacy=True) once the v2 PRs add that flag, and plain run() until then.
      • Compared exactly: cluster metadata (reads, UMIs, cluster fraction, consensus, filter pass), consensus sequences, pairing decisions and loci.
      • V/J calls, junction and productive are compared exactly only when the installed abstar matches the golden version; otherwise that check is skipped. Germline database updates can legitimately change calls, and CI installs the latest abstar.
    • The default (new) mode must run end to end and produce pairs. If it writes pairs/pairs.parquet, the file must have the contract columns.
    • The fixture itself is checked to still exercise pairing and filters.
  • make_golden.py: regenerates the tables and records the commit and abstar version. Use it only for an intended, reviewed change.

Verified

  • Deterministic: 7/7 tests pass on two consecutive runs, about 50 s with 2 workers.
  • Catches changes: min_cluster_reads=5 changes metadata, consensus and pairs; clustering_threshold=0.6 changes metadata and consensus. Both fail the test.
  • Full suite: pytest pairplex 33 passed.

Effect on the open v2 PRs

Each v2 branch adds legacy=True. Once this test is on main, CI proves on every PR that legacy mode still reproduces the reference output.

🤖 Generated with Claude Code

Two simulated wells (<1000 reads each, so one deterministic chunk) run through the real PairPlex +
abstar pipeline. Reference behaviour (legacy=True once that flag exists, plain run() before) must
equal the stored golden tables: cluster metadata, consensus, pairing decisions and loci exactly;
annotation calls exactly only when the installed abstar matches the golden version. The default
mode must run and produce well-formed outputs (and honour the pairs/pairs.parquet contract when
it writes one). Regenerate with pairplex/tests/data/golden/make_golden.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bnemoz
bnemoz merged commit 2726c34 into main Nov 17, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant