Outcome metrics ndcg/recall/map collapse to discounted-HR / HR / MRR under leave-last-out — decide whether to keep all three #13

Summary

Under our leave-last-out split there is exactly one held-out positive per user (_assign_indices_and_split). In that regime the three "outcome" ranking metrics in _single_relevant_metrics each degenerate to a monotone function of the single positive's rank, so they are three views of the same hit-rank distribution rather than three independent signals:

• ndcg = 1/log2(rank+2) if the positive is in the top-k else 0 → a discounted hit rate.
• recall = hit_any.mean() → exactly HR@k (recall = hits / relevant = hits / 1).
• map = mean(1/(rank+1)) → exactly MRR (average precision with one relevant item is its reciprocal rank).

None of the three exercises what NDCG/MAP/recall normally buy over a hit metric (graded relevance, multiple relevant items) because there is only one binary positive.

Why this is a simplification decision

Per the Simplicity Criterion in CLAUDE.md, carrying three metrics that are monotone transforms of one another is redundant surface: three columns in every trajectory/result, three things a reader assumes are independent axes when they are not. Options, roughly in order of my preference:

1. Keep only ndcg as the single outcome metric (it is already the HPO objective) and drop recall/map. Smallest, most honest surface; a net-negative diff.
2. Keep all three but document that they collapse to discounted-HR / HR / MRR under leave-last-out, so no one mistakes them for independent signals. Cheapest, but keeps the redundancy.
3. Keep all three and rename to hr / mrr to reflect what they actually are in this regime. Honest naming, but more churn (schema, trajectory docs, downstream readers) than option 1.

Note this is scoped to the outcome trio only. train_ndcg is a genuine multi-relevant NDCG and is unaffected; the diversity/popularity/model-intrinsic metrics never touch the held-out set.

Also worth deciding here (related, lower priority)

train_test_ndcg_gap subtracts a single-positive held-out NDCG (tiny, IDCG=1) from a multi-relevant train_ndcg (properly normalized, can approach 1). The two terms are not on the same scale, so the gap is dominated by the train term and is only meaningful as a relative trend across configs. If we rework the outcome metrics we should decide whether the gap definition still makes sense.

Acceptance

• A decision recorded (in code comments / module docstring) on which outcome metrics to keep and why, given the one-positive-per-user split.
• If metrics are removed/renamed: update TRAJECTORY_SCHEMA.md, any result readers, and tests accordingly.