Skip to content

Fix per-allocator statistics in Bencher reports - #271

Merged
jvoisin merged 1 commit into
daanx:masterfrom
dajiaohuang:fix/bencher-per-allocator-statistics-20261015
Oct 17, 2026
Merged

jvoisin merged 1 commit into
daanx:masterfrom
dajiaohuang:fix/bencher-per-allocator-statistics-20261015

Conversation

@dajiaohuang

Copy link
Copy Markdown
Contributor

Problem

Running python scripts/bencher.dev.py benchres.csv on results from multiple allocators writes the same cross-allocator statistics into every allocator's JSON file. For example, cfrac times of 1/3 seconds for sys and 10/14 seconds for mi produce a mean of 7 seconds in both files, instead of 2 and 12 seconds respectively. Memory mean/min/max are mixed in the same way.

The allocator enumeration also uses the last parsed benchmark, so allocators that only ran other benchmarks disappear from the output.

Change

Enumerate allocators from the full input and compute each benchmark's statistics using only that allocator's samples. Omit benchmark/allocator combinations with no samples. Preserve the existing output schema and filenames.

Validation

  • python -m unittest discover -s scripts -p test_bencher.py -v: two CLI regression tests fail on the base implementation and pass with this change (all six statistics and differing benchmark selections).
  • git diff --check passes.
  • Tests use small synthetic benchres.csv fixtures, not measured performance comparisons. No full allocator builds or stress benchmarks were run.

@jvoisin
jvoisin merged commit 3913826 into daanx:master Oct 17, 2026
77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants