Skip to content
TwinCommitDocs

Apache Hive

TwinCommit applies changes to Apache Hive in one of two modes. For Hive ACID transactional tables it writes ORC delta files and commits them with MERGE. For Iceberg tables registered in the Hive Metastore it commits Iceberg snapshots. In both modes the checkpoint is part of the same commit.

Supported: Hive 3.1 and 4.x; Hive Metastore with Iceberg 1.4+.

Configuration

Section titled “Configuration”
target:
  type: hive
  format: acid               # acid | iceberg
  jdbc_url: jdbc:hive2://hs2.example.com:10000/replica;transportMode=http;httpPath=cliservice
  metastore_uri: thrift://hms.example.com:9083
  warehouse: hdfs://nameservice1/warehouse/replica
  staging: hdfs://nameservice1/tmp/twincommit
  auth:
    kerberos_principal: twincommit/_HOST@EXAMPLE.COM
    keytab: /etc/security/keytabs/twincommit.keytab
  merge_interval: 30s

Requirements

Section titled “Requirements”
  • ACID mode: hive.support.concurrency=true and hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager; tables created STORED AS ORC TBLPROPERTIES ('transactional'='true'). twincommit ddl -apply creates them this way.
  • Iceberg mode: an Iceberg-enabled metastore and write access to the warehouse path.
  • Write access to staging.

Grants (Ranger or SQL standard authorization)

Section titled “Grants (Ranger or SQL standard authorization)”
GRANT ALL ON DATABASE replica TO USER twincommit;