Skip to content

Add native SQLite and PostgreSQL execution storage - #67

Open
pallaoro wants to merge 4 commits into
mainfrom
native-sql-storage
Open

pallaoro wants to merge 4 commits into
mainfrom
native-sql-storage

Conversation

@pallaoro

@pallaoro pallaoro commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

ClawFlow currently persists execution state only as JSON files. This adds explicit SQLite and PostgreSQL backends for checkpoints, memoized outputs, traces, running-step state and pending approvals, while keeping JSON as the default.

  • Configure storage.backend: "sqlite" (Node 22.13+, built-in SQLite/WAL) or "postgres" with connectionStringEnv and a stable namespace. Initialization is lazy and versioned; a database failure never falls back to files.
  • Runner, plugin status reads and retention use an async-compatible store. SQL commits checkpoints/memos/approvals atomically, claims resumes transactionally, lists indexed summaries, and scopes cancellation/events by store. Parent retries reuse completed child checkpoints.
  • Add paged descendant enumeration (listSubRuns) with actual IDs and authenticated HTTP run reads/approval actions. Approval acknowledges only after its durable claim; SQL storage errors return 503.
  • Optional includeOldest: true / include_oldest=1 returns the oldest matching checkpoint ID/timestamp before pagination. SQL count, metadata and page share one read transaction; child runs are excluded from root diagnostics.
  • Export createStateStore and importJsonState; provide an explicit offline JSON migration command that preserves source files and rejects conflicts. README covers configuration, migration, backup, permissions and boundaries.

Validation: build and all 302 tests passed on Node 22.22.3 and Node 24.22.2 with native SQLite and an isolated real PostgreSQL 15 instance. The current head passes CI on both majors with PostgreSQL 16. Includes restart/resume, concurrent mutations/resumes, transaction rollback, nested retries, pagination, retention, migration, plugin reads, cancellation races, event isolation and database-unavailable behavior. The documented migration CLI was exercised and its result read through built exports. CI now provisions PostgreSQL 16 and runs the same suite.

Rollout: this does not switch installations, change the Clawnify pin or deploy anything. Flow definitions, versions, trigger schedules and memory files remain in the workspace. Managed hook/CAP/sleep integration is reviewed in https://github.com/clawnify/clawnify/pull/2814; enabling SQL still requires that integration and a backup/restore canary. Execution storage remains separate from central history. PostgreSQL namespaces isolate keys, not permissions; operators must use database roles/schema grants for security isolation.

Dependency audit still reports the unchanged baseline optional openclaw@0.0.1 and development esbuild@0.27.4 advisories; no new PostgreSQL driver advisory was reported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant