All pull requests

Pull requests

• [CI] Make dataset suffix hooks run on Windows

  #2733·LittleChenLiya opened Nov 14, 2026·

  ·
  1

  

• [Fix] Stop MATH is_equiv from equating different answers

  #2732·LittleChenLiya opened Nov 14, 2026·

  ·
  

• [Fix] Read the StrategyQA verdict from the answer end

  #2731·LittleChenLiya opened Nov 14, 2026·3 of 5 tasks·

  ·
  

• [Fix] Read LawBench, OpenFinData, CMO, AlignBench and Flames data as UTF-8

  #2730·Zephyr4772 opened Nov 13, 2026·5 of 6 tasks·

  ·
  

• Fix [Th]he case-sensitivity typo in TabMWP answer extraction

  #2729·selimfedakar opened Nov 12, 2026Contributor·

  ·
  

• [Fix] 修复 Agent 首次推理启动失败及串行 Chat/Agent 的 JSONL 断点恢复

  #2728·kernelvale opened Nov 12, 2026·3 of 5 tasks·

  ·

• Fix str.find() truthiness in MATH InternLM answer postprocessor

  #2727·selimfedakar opened Nov 11, 2026Contributor·

  ·

• [Fix] Stop first_option_postprocess from extracting option letters inside words

  #2726·shlokbam opened Nov 11, 2026·3 of 6 tasks·

  ·
  1

• [Fix] Make HumanEval evaluator compatible with human-eval without ignore_incomplete

  #2724·julio75012 opened Nov 11, 2026·

  ·

• [Fix] Limit LiveCodeBench time-split generations to 1024 tokens

  #2723·julio75012 opened Nov 11, 2026·

  ·

• [Fix] Use UTF-8 for result station JSON

  #2722·anticipate218 opened Nov 10, 2026·4 of 6 tasks·

  ·

• [Fix] Correct multi_choice answer-marker guard in GaokaoBench and Kaoshi

  #2721·selimfedakar opened Nov 10, 2026Contributor·

  ·

• [Fix] Keep cleaned text in general_cn_postprocess

  #2720·aidless opened Nov 8, 2026·3 tasks done·

  ·
  

• [Fix] Preserve metric priority in summarizer groups and tables

  #2719·feiiiiii5 opened Nov 6, 2026·3 of 6 tasks·

  ·
  1

  

• [Fix] Preserve literal placeholders in prompt field values

  #2718·feiiiiii5 opened Nov 6, 2026·4 of 6 tasks·

  ·
  1

  

• [Fix] MedXpertQA: return empty sentinel when no option letter is found

  #2712·selimfedakar opened Nov 3, 2026Contributor·

  ·
  

• Fix sliced dataset sizes on partitioner cache misses

  #2711·Cookie0103 opened Nov 3, 2026·4 of 6 tasks·

  ·
  

• [Fix] CHHallu-Src: parse the model's answer object, not a greedy brace span

  #2710·selimfedakar opened Nov 2, 2026Contributor·

  ·
  

• [Fix] Ignore empty-normalized references in TriviaQA/NQ substring match

  #2709·selimfedakar opened Nov 2, 2026Contributor·3 of 6 tasks·

  ·
  1

  

• [Fix] Preserve compression summary headers during pivot conversion

  #2706·yunyv opened Nov 1, 2026·3 of 6 tasks·

  ·

• [Fix] Print repeat-analysis summaries when no predictions are available

  #2703·yunyv opened Oct 31, 2026·3 of 6 tasks·

  ·

• [Fix] Stop sending temperature in ClaudeSDK requests

  #2702·pcbeingused333 opened Oct 29, 2026·

  ·
  1

  

• [Fix] Preserve subdivision names in repeated evaluation metrics

  #2701·xlb3055 opened Oct 28, 2026·2 of 4 tasks·

  ·

• [Fix] Average n runs when the evaluator returns no details

  #2699·FazeelUsmani opened Oct 27, 2026·3 of 6 tasks·

  ·
  

• [Fix] CascadeEvaluator miscounts judge replies starting with 'Correctness'/'Correctly' as correct - #2657

  #2697·lemonzoey opened Oct 26, 2026·3 of 4 tasks·

  ·
  1