Repository navigation
9ff3f391: reconcile upload accepts Finom CSV statements - #342
Merged
Merged
Conversation
The reconcile flow could parse Finom CSV exports only through the email intake; the CLI, portal API and document upload path were PDF-only at every layer, so a Finom statement CSV had no way into reconciliation. - recognizeStatement now recognizes tabular exports from their header row against the same fixed layouts the email intake already uses (Finom statement, Revolut), including the selected-account mismatch check; the email intake keeps its behavior, with errors that now carry the filename. - prepareStatementImport commits an identity-less tabular statement only when the operator explicitly selected the account (--account), which stands in for the missing IBAN; without it the source stays at review, as before. Only a source with header identity refuses to coexist with an unverified overlapping source, so re-uploads of the same file replay to the same statement id. - Bank-statement documents accept text/csv at prepare and completion (the object must be readable text, not a renamed PDF or binary); XLSX and every other document type stay PDF-only. - dataops reconcile upload (alias: upload-statement) accepts statement.pdf or statement.csv, validating bytes locally before any upload; help documents the --account rule for CSV exports. Scope: in
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task 9ff3f391 — "CLI gap: reconcile upload Finom statement".
Problem
The reconcile flow parses Finom CSV exports only through the email intake (
recognizeEmailStatement, backend/src/reconcile/statementIntake.ts). Every explicit upload layer was PDF-only, so a Finom statement CSV had no way into reconciliation except emailing it to the invoice address:contentTypeFormapped only.pdf(cli/src/documentUpload.mjs)dataops reconcile uploadranrequirePdfBytesunconditionally (cli/src/commands/reconcile.mjs)recognizeStatementaccepted PDFs only (backend/src/reconcile/statementIntake.ts), pinned by testsChange
recognizeStatementnow recognizes tabular exports from their header row against the same fixed layouts the email intake uses (FIXED_LAYOUTS.finom:Completed date, Counterparty name, Reference, Amount, Balance— already in repo with a synthetic fixture) plus Revolut, with the selected-account mismatch check. The email intake path is behavior-identical (it now callsrecognizeStatementdirectly; errors carry the filename).prepareStatementImportcommits such a statement only when the operator explicitly selected the account (--account finom), which stands in for the header identity; without it the source stays at review exactly as the email intake does (StatementIdentityReviewError). Rows key asunverified:<statementId>per the existing selection machinery, so dedup on re-upload follows the repo's existing conventions: same file replays to the same statement id (statementIdFor(documentId, account, period), preview says "no differences" and requires--replace); overlapping rows from a different file are kept once (earliest upload wins) byrowKeys(date+amount+currency+counterparty+reference, occurrence-numbered). Only a statement that has header identity refuses to coexist with an unverified overlapping source (clearer message now).text/csvat prepare and completion; the object must be readable text (no NUL bytes, no PDF signature mismatch). XLSX and every other document type stay PDF-only.dataops reconcile upload --file statement.pdf|statement.csv(upload-statementregistered as an alias) validates bytes locally, stores the document and imports it; help documents the CSV/--accountrule.Tests (targeted only, all synthetic)
backend/tests/reconcile.test.ts: Finom CSV import end to end on an isolated synthetic month (2029-10) — review without account, read-only parse, parse with wrong account, import with account (7 rows, EUR closing balance), idempotent replay (preview identical →--replacekeeps the same statement id and rowCount), balance lookup, deduped overlapping second export, unknown-CSV refusal, legacy CSV-backed rows readable.backend/tests/bookkeeping-document-atomic-api.test.ts: text/csv prepare+complete succeeds; XLSX still refused at prepare; declared-PDF object with CSV bytes still refused at completion.cli/test/reconcile.test.mjs: CSV upload passestext/csvthrough prepare and imports under the named account; XLSX / PDF-mislabeled / binary-.csv / receipt-.csv refusals keep usage exit codes.Results
tsc --noEmitcleanRead-only toward prod; no real financial data (fixtures are synthetic, invented counterparties). Frontend portal upload stays PDF-only (unchanged) — a portal CSV flow can follow the same backend path later.