ScribeAI docs
Guide 084 min read

Durable-format and process compatibility

Version, migrate, reject, preserve, and recover durable formats and packaged process protocols.

Reflects the current codebase View Markdown

Scribe versions each durable JSON format independently from SQLite and requires exact compatibility between packaged processes. The executable registry is src/contracts/compatibility.ts; this document is the release and recovery policy.

Alpha policy#

Until the first compatibility-supported release, wire and durable JSON contracts remain version 1 and are replaced in place without dual-read paths or migrations. Test fixtures describe only the current alpha shape. Developers must remove the existing application user data before running a build whose alpha contracts changed; old alpha data is not supported.

Supported versions#

Boundary Format/protocol name Current Minimum readable Minimum migratable Newer-version behaviour
SQLite scribe.sqlite 6 6 6 Open only for inspection, reject startup, preserve the file.
Block document scribe.blocks 1 0 0 Preserve and quarantine; do not overwrite; document hydration is unavailable.
Suggestion command result scribe.suggestion-command-result 1 0 0 Preserve and quarantine; reject the command lookup.
Suggestion fact suggestion_event_history.event_version 1 — — Preserve and quarantine; stop projection at that sequence. Later facts are not applied.
Suggestion projection scribe.suggestion-projection 1 0 0 Preserve and quarantine; suggestion hydration is unavailable.
Pi loop entry scribe.pi.loop-state 1 0 0 Preserve the Pi session and disable autonomous resume.
Storage process scribe.storage 1 1 1 Reject readiness before any request.
Agent process scribe.agent 1 1 1 Reject readiness before any request.

Version 0 means the legacy unwrapped JSON shipped before explicit envelopes. Successful reads validate and upgrade it once. Current writes use named envelopes; document blocks specifically use { format, version, blocks }, while projections, events, and command results use named payload fields. Transforms are pure, sequential, and must provide every contiguous version edge.

Preservation and quarantine#

The durable_json_quarantine table records the original JSON text, format name, record identity, detected version, stable error code, and timestamp. A unique format/identity pair prevents duplicate records. Active reads never cast quarantined content into runtime state. Normal logs and errors identify the feature and record but do not include its content.

Block JSON is the canonical draft data. An unsupported format error tells the caller which feature is unavailable, identifies the database quarantine location, and requires a newer application. Unknown Pi entries not owned by Scribe are left entirely to Pi and are not modified.

Process readiness#

Every utility-process ready message contains the protocol name, exact version, build identifier, and complete supported operation set. Main compares all four values with its own registry. A version, build, protocol-name, or operation-set mismatch fails readiness with PROTOCOL_VERSION_MISMATCH; a structurally invalid ready message fails with MALFORMED_READY_HANDSHAKE. No RPC request is sent before this check succeeds, and arbitrary cross-version feature negotiation is not attempted.

Release checklist#

Before every release:

  1. Review the SQLite schema, event payloads, BlockNote serialization, suggestion projection and command result, Pi Scribe entries, and both operation registries.
  2. During alpha, replace durable shapes in place at version 1 and recreate user data. After compatibility support begins, increment only the changed format version, add a pure contiguous migration, retain old fixtures unchanged, and add a new current fixture.
  3. During alpha, keep wire protocols at version 1 and update current fixtures in place. After compatibility support begins, increment the protocol version for registry or wire changes. Storage and agent operation sets must always match their ready handshakes.
  4. Test oldest, every intermediate, current, future, invalid, failed-transform, and idempotent legacy-upgrade cases. Verify future/invalid source text remains preserved exactly once.
  5. Test a future event followed by a known event and confirm projection stops at the gap until snapshot recovery.
  6. During alpha, bump the database version and recreate the database. Do not add a migration until a release explicitly commits to database compatibility.
  7. Run npm test, npm run lint, npm run build, and npm run docs:build.

Do not regenerate a historical fixture with a current serializer. Fixtures under src/contracts/fixtures/compatibility/ are immutable after release.

Backup and rollback#

Downgrade writes are unsupported. To return to an older application after migration:

  1. Close Scribe completely.
  2. Preserve the newer scribe.sqlite3 and its WAL/SHM files as a recovery copy.
  3. Locate the pre-migration .bak beside the database and copy it into place as scribe.sqlite3.
  4. Run the application release whose schema version matches that backup.

Never point an older application at the migrated database. Never automate rollback by deleting or rewriting the newer database.