Configuration reference

A TwinCommit pipeline is one YAML file that names a source, a target and the tables to replicate. Every value can reference environment variables with ${VAR}, so credentials never need to be stored in the file.

Complete example

source:
  type: db2
  database: PRODDB
  user: ${DB2_USER}
  password: ${DB2_PASSWORD}

target:
  type: postgres
  dsn: postgres://twincommit:${PG_PASSWORD}@pghost:5432/analytics

tables:
  - source: DB2INST1.ORDERS
    target: sales.orders
  - source: DB2INST1.CUSTOMERS
    exclude_columns: [NOTES]

state_schema: twincommit
metrics_addr: ":9108"

batch:
  max_rows: 5000
  flush_interval: 1s
  poll_interval: 500ms
  max_txn_memory_bytes: 268435456

spill_dir: /var/lib/twincommit/spill

control:
  token: ${TWINCOMMIT_CONTROL_TOKEN}
  allow_writes: false

`source`

Key	Type	Description
`type`	string	`oracle`, `mysql`, `sqlserver`, `postgres`, `mongodb`, `db2`, `db2zos`, `cassandra`, `mariadb` or `saphana`
connection keys		Depend on `type`. See the source pages, for example MySQL, MongoDB or Db2 LUW.

`target`

Key	Type	Description
`type`	string	Relational: `oracle`, `mysql`, `sqlserver`, `postgres`, `db2`, `mariadb`, `saphana`. Warehouse and lakehouse: `snowflake`, `databricks`, `synapse`, `redshift`, `bigquery`, `spark`, `hive`. Data lake: `adls`, `s3`, `gcs`. NoSQL, search and streaming: `mongodb`, `cassandra`, `redis`, `elasticsearch`, `kafka`
connection keys		Depend on `type`. See the target pages.

`tables`

A list of tables to replicate.

Key	Type	Default	Description
`source`	string	required	`SCHEMA.TABLE` on the source
`target`	string	lower-cased source name	`schema.table` on the target
`exclude_columns`	list	`[]`	Columns not to replicate, such as unsupported LOB types or sensitive data

Top-level keys

Key	Type	Default	Description
`state_schema`	string	`twincommit`	Target schema for checkpoint and table-state tables
`metrics_addr`	string	`:9108`	Address for `/metrics`, `/status` and `/control`
`spill_dir`	path	`/tmp`	Where transactions over `max_txn_memory_bytes` are spilled

`batch`

Key	Type	Default	Description
`max_rows`	int	`5000`	Apply once this many row changes are pending
`flush_interval`	duration	`1s`	…or after this long
`poll_interval`	duration	`500ms`	Wait between log reads when caught up
`max_txn_memory_bytes`	int	`268435456` (256 MiB)	Larger transactions spill to `spill_dir`

Lower flush_interval for lower latency; raise max_rows for higher throughput on targets with expensive commits, such as lakehouse targets.

`control`

Key	Type	Default	Description
`token`	string	none	Bearer token required on `/status` and `/control`
`allow_writes`	bool	`false`	Allow start, stop and re-snapshot through `/control`, used by the MCP server
`protect_metrics`	bool	`false`	Also require the token on `/metrics`

Environment substitution

${VAR} is replaced with the value of the environment variable VAR when the file is loaded. An unset variable becomes an empty string, and twincommit check warns about it.