Architecture

TwinCommit is one process with three stages: a source that reads the database log, an agent that groups changes into transactions and orders them, and a target that applies them and stores the checkpoint. The source and target never talk to each other directly. Everything passing between them is a product-neutral model of tables, rows, transactions and log positions.

                 SOURCE                    AGENT                          TARGET
 ┌──────────┐   ┌────────────┐   ┌───────────────────────┐   ┌──────────────────────────┐
 │ redo /   │──▶│ log reader │──▶│ transaction buffer    │──▶│ apply (upsert / delete)  │
 │ recovery │   │ + row      │   │  ├─ in memory         │   │   + checkpoint row       │
 │ log      │   │   decoder  │   │  └─ spill_dir (large) │   │   in ONE transaction     │
 └──────────┘   └────────────┘   │ commit-ordered queue  │   └──────────────────────────┘
                                 └───────────────────────┘
 ┌──────────┐   ┌────────────┐                               ┌──────────────────────────┐
 │ tables   │──▶│ snapshot   │──────────────────────────────▶│ bulk copy (COPY / MERGE) │
 └──────────┘   └────────────┘                               └──────────────────────────┘

1. Log reader

Each source has a native log reader:

Source	How changes are read
Db2 LUW / z/OS	The Db2 log read API; row images are decoded by TwinCommit
Oracle	Online redo and archived logs, all RAC threads, merged in SCN order
SQL Server	The active and backed-up transaction log
PostgreSQL	A logical replication slot using pgoutput

Nothing is installed in the source: no triggers, change tables or stored procedures.

2. Transaction buffer

Log records are grouped by source transaction. Nothing is applied until the commit record is read. A rollback, or a rollback to a savepoint, discards the affected changes, so they never reach the target.

Transactions larger than batch.max_txn_memory_bytes are written to spill_dir and read back at commit. See spill to disk.

3. Commit-ordered apply

Committed transactions are applied in source commit order. Small transactions are batched together, up to batch.max_rows row changes or batch.flush_interval, whichever comes first. Every change is applied as an upsert or a delete by primary key, so re-applying log the target already contains is harmless.

4. Checkpoint in the target

The log position of the last applied transaction is stored in a table in state_schema on the target and written in the same transaction as the data. After a crash, TwinCommit resumes from exactly that position. See exactly-once delivery and checkpoints.

5. Snapshot handoff

The initial load takes a brief share lock per table, records the log position while no writer is mid-transaction, releases the lock and bulk-copies the rows. Streaming starts from the recorded position, and idempotent upserts make the overlap safe. See snapshots.

Observability

While twincommit run is active it serves /metrics (Prometheus) and /status (JSON) on metrics_addr, and writes JSON log lines to stderr. The MCP server reads the same /status endpoint.