Skip to content

xcsqa-eu: primary metric acc, and changelog entry for #142 - #150

Merged
HannanJaved merged 3 commits into
mainfrom
fix-xcsqa-metric
Nov 3, 2026
Merged

HannanJaved merged 3 commits into
mainfrom
fix-xcsqa-metric

Conversation

@haideraltahan

Copy link
Copy Markdown
Collaborator

#142 removed acc_norm from the XCSQA task's metric_list, but xcsqa-eu in task-groups.yaml still names acc_norm as its primary metric, and its description still says 0-shot. collect finds no acc_norm in the results and falls back to every metric, so XCSQA comes out as acc, acc_stderr and sample_len rows instead of one acc row.

This sets metric: acc, updates the description to 10-shot, and adds the CHANGELOG entry #142 missed, plus a "Fixed" line for the metric. Scores don't change. tests/test_task_groups.py, tests/test_primary_metrics.py and tests/test_collect_results.py pass.

@HannanJaved
HannanJaved merged commit c5dea69 into main Nov 3, 2026
3 checks passed
@HannanJaved
HannanJaved deleted the fix-xcsqa-metric branch November 3, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants