Apache Spark
real-time ingestion
TwinCommit applies changes to Apache Spark with exactly-once, commit-ordered semantics from any of 9 log-based sources, including Oracle, MySQL, SQL Server and Db2. A Structured Streaming source for exactly-once change feeds.
Apache Spark
Apache Spark 3.5 and 4.x · Scala 2.13 · PySpark
- ✓The twincommit-spark connector JAR on the driver class path
- ✓Network access from the driver to the TwinCommit change feed endpoint
Apply
How TwinCommit writes to Spark
The TwinCommit Spark connector exposes each pipeline as a Structured Streaming source with offsets mapped to log positions, so your jobs get exactly-once semantics with Spark checkpoints.
Spark target docs →Spark connector FAQ
Which Spark versions does TwinCommit support?
+
Apache Spark 3.5 and 4.x · Scala 2.13 · PySpark.
How does TwinCommit write to Spark?
+
The TwinCommit Spark connector exposes each pipeline as a Structured Streaming source with offsets mapped to log positions, so your jobs get exactly-once semantics with Spark checkpoints.
Replicate your first table in five minutes.
Download the free Developer edition, or have one of our replication engineers walk you through a migration plan.