Repository navigation
Open
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation
Description
Summary
The bulk-recompute runbook tooling and two code-review reports exist only in the working tree. No commit contains them, and nothing under docs/ references the runner script, so the bulk recompute rollout cannot be reproduced from a clean checkout.
Location
- Doc:
docs/deployment-guide.md(runbook section, currently silent on bulk recompute) anddocs/UPDATE_SUMMARY.md - Source of truth:
apps/web/app/routes/api.admin.recompute.ts:1-12(the endpoint's own usage header) andscripts/run-recompute.sh
Problem
git status --porcelain reports these paths as untracked:
?? docs/UPDATE_SUMMARY.md
?? plans/reports/code-review-260305-skill-references-scripts.md
?? plans/reports/code-reviewer-260306-1531-admin-recompute-endpoint.md
?? scripts/run-recompute.sh
scripts/run-recompute.shis the operational driver for the recompute endpoint: it defaultsAPI_URLtohttps://skillx.sh/api/admin/recompute, requiresADMIN_SECRET, takes a batch size argument, and supports resume by re-running.grep -rn 'run-recompute' --include=*.md --include=*.json .excludingnode_modulesreturns no matches, so the script is unreferenced and a maintainer has no documented way to find it.docs/UPDATE_SUMMARY.mddescribes the Mar 5 documentation pass and overlapsdocs/project-roadmap.md, which already records the same Phase 3.1-3.3 completions.
Proposed content
- Commit
scripts/run-recompute.shand add a short runbook subsection (deployment guide) covering the requiredADMIN_SECRET, thebatchargument, resume behavior, and theall=truebatch cap. - Decide the fate of
docs/UPDATE_SUMMARY.md: keep it as a dated record or delete it as duplicated bydocs/project-roadmap.md. - Commit the two review reports under
plans/reports/, since the recompute hardening issues cite them as their evidence source.
Acceptance criteria
-
scripts/run-recompute.shis tracked in git and referenced from a docs page -
docs/UPDATE_SUMMARY.mdis either committed or deleted - The two
plans/reports/review reports are committed or deleted - Documentation matches live behavior (verified against
apps/web/app/routes/api.admin.recompute.ts)
Notes
- Two further untracked entries,
.claude/session-state/andapps/web/.claude/session-state/, are session state and should stay ignored — they are out of scope here. - The review reports are the evidence base for the recompute hardening issues, so deleting them would remove the cited source unless it is summarised into the issues first.
- Related: PR feat: skill references & scripts support #9 merged the recompute endpoint itself; this issue only closes the gap between the merged code and its operational instructions.
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation
Activity
clark-cant commentedon Oct 31, 2026
Verified on
mainatc95ab77— 3 of the 4 acceptance criteria are already met; the runbook subsection is the only real work leftWhat is done
scripts/run-recompute.shtracked in gitacfb24f(2026-10-14, #83 "accept reviewed session-derived overlay proposals"), i.e. the overlay PR carried it, which is why this gap stayed invisiblescripts/README.md:125-139has a "Leaderboard Recompute (Operations)" section;docs/deployment-guide.mdstill has no recompute entrydocs/UPDATE_SUMMARY.mdcommitted or deletedgit log --all -- docs/UPDATE_SUMMARY.md(empty), so it never entered history and nothing danglesplans/reports/reviews committedcode-review-260305-skill-references-scripts.md(9,020 B) andcode-review-260306-1531-admin-recompute-endpoint.md(7,629 B)Where the docs still disagree with the endpoint (verified against
apps/web/app/routes/api.admin.recompute.tsatc95ab77)POST … — run one batch (default 100),batch=200,offset,resume=trueandall=true, but never mentionMAX_BATCHES_PER_REQUEST = 50(line 23) or any upper bound onbatch. The header is what a curl user reads first, and it is less complete thanscripts/README.md, which does state the 50-batch cap. The two docs contradict each other in coverage.parseIntSafeforparseBatchSize, clampingbatchto 1..500 with fallback 100 (apps/web/app/lib/admin/batch-limit.ts, constantsRECOMPUTE_BATCH_LIMIT). After it lands,batch=0and negatives silently become 100 and anything above 500 is clamped — undocumented today. Whatever numbers the runbook publishes should be taken from that module, not retyped.run-recompute.shinterpolatesBATCH_SIZE="${1:-10}"straight into?batch=. Today./scripts/run-recompute.sh 0reproduces the fix(admin): batch param is unclamped and batch=0 spins 50 no-op batches in all=true mode #28 spin (50 no-op batches,nextOffsetnever advances); after fix(admin): clamp the recompute batch size to 1..500 #169 the same command silently becomes 100. The README's "larger batches" example is fine, but it should state the 1..500 bound.resume=trueon its very first call — so two overlapping invocations silently share one checkpoint and can double-score rows. One line ("run a single instance at a time until fix(leaderboard): bulk recompute has no concurrency guard, overlapping all=true runs race on the KV checkpoint #26 lands") is the whole fix.all=trueis a different code path. The script loops client-side over single batches and never sendsall=true, so "a singleall=truerequest is capped at 50 batches" describes the endpoint, not the script. Worth splitting into two sentences so the 50×batch ceiling is not misread as a ceiling on the script.Narrowed scope — one docs-only PR
### Recompute Leaderboard Scoresblock under## Common Tasksindocs/deployment-guide.md(the section already carries Update Seed Data / Update Code / Add Environment Variable / Database Backup at lines 703-757). It should cover:ADMIN_SECRETrequired,batchrange 1..500 default 100 (post-fix(admin): clamp the recompute batch size to 1..500 #169),offset,resume=true, single-instance-at-a-time, and the 50-batchall=truecap.RECOMPUTE_BATCH_LIMITso the numbers have one source.No work in flight: no open PR references #29; #169 and #170 touch the recompute path but neither adds documentation.
Generated by the github-maintain skill at 2026-10-31T16:07Z