RonDB training · module 09 of 25 · 18 min

Transactions and concurrency

Locks, isolation, the commit protocol that both commits and replicates, and what happens when nodes fail or transactions deadlock.

In one slide · RonDB's concurrency model

RonDB concurrency: read committed, row locks, and a commit protocol that replicates

ChoiceRonDBWhy
IsolationRead committed by defaultReaders never wait; no memory for snapshots
Concurrency controlPessimistic: strict two-phase row lockingPredictable latency under conflicts, simpler recovery
Range locksNoneWould need locks on every node of a hash-partitioned table
CommitRonDB's own two-phase commitAtomicity and replication in one protocol, non-blocking
DeadlocksTimeout; proactive detection optionalSafe and simple; also catches stalls from node failures

docs: rondb_concurrency_control.md (TLDR; RonDB Specs, RonDB's Choice) · rondb_nonblocking_2pc.md · code: ConfigInfo.cpp (EnableProactiveDeadlockDetection)

Isolation · what each level allows

Isolation: read committed by default; stronger reads by asking for locks

LevelPreventsIn RonDB
Read uncommittednothingNot supported: no dirty reads
Read committeddirty readsThe default: always the latest committed value
Repeatable read+ non-repeatable readsWith shared or exclusive row locks, as the application asks
Serializable+ phantomsNot for ranges: no range locks. Lock a row that stands for the range

A long scan under read committed is not a snapshot: each row is the latest committed version when the scan reaches it.

docs: rondb_concurrency_control.md (Isolation Levels, Range Locks, Range Lock Workarounds)

Lock modes · SQL and the NDB API

Lock modes: each read chooses its lock; every write takes an exclusive one

SQLNDB APILockBehaviour
SELECT … (default)LM_CommittedReadnoneReads the committed row in DataMemory, even while it is being written
SELECT … FOR SHARELM_ReadsharedMany readers or one writer; held to the end of the transaction
SELECT … FOR UPDATELM_ExclusiveexclusiveOne reader or writer; no lock upgrade later
—LM_SimpleReadshared, brieflyReleased as soon as the row is read
INSERT · UPDATE · DELETE—exclusiveHeld until the transaction completes

Read committed with writes in flight is WORM locking, write once read many: the writer works on its copy in TransactionMemory (module 07).

code: include/ndbapi/NdbOperation.hpp (enum LockMode) · sql/sql_yacc.yy (locking_clause) · docs: rondb_concurrency_control.md (Supporting Read Committed)

Strict 2PL · locks grow, then shrink

Strict two-phase locking: locks are taken per operation, released at commit

prepare: every read and write, one operation at a time commit complete A · shared B · exclusive C · shared D · exclusive shared locks releasedshared locks released on backups until complete

The MySQL server sends one operation at a time; each takes the lock it needs. All reads and writes happen in the prepare phase.

Shared locks go in the commit phase.

Exclusive locks go at commit on the primary, at complete on the backups: strict two-phase locking. Gradual locking leaves room for application deadlocks.

docs: rondb_nonblocking_2pc.md (RonDB's 2PC, Complete Stage) · code: dblqh/DblqhMain.cpp (commitReqLab: primary at COMMIT; execCOMPLETE: backups) · the timeline is an illustration

A classic deadlock · two shared locks, two upgrades

Upgrade deadlock: two readers that both decide to write wait for each other

row Xshared: T1, T2 T1SELECT … FOR SHARE T2SELECT … FOR SHARE T1: UPDATE X · waits for T2's shared lock T2: UPDATE X · waits for T1's

Both transactions hold a shared lock on the same row. That is fine for reading.

Both then update it. Neither shared lock can become exclusive while the other exists: a deadlock. If you will write, read with FOR UPDATE.

docs: rondb_concurrency_control.md (Use-case Exclusive Locks)

Special cases · where read committed takes a lock

Read committed exceptions: unique key lookups and BLOB reads lock briefly

Lookup by unique key

  • Takes a shared lock on the unique index row until the base row is read, so a delete and re-insert cannot fool it.
  • With locks or writes, it locks index row then base row; a write by primary key locks the other way: deadlock risk.
  • Use unique keys as constraints, not as lookup paths in write transactions.

Reading a BLOB

  • Its parts are several rows, so a lockless read could mix versions.
  • A shared lock on the base row is held while the parts are read, then released.
  • BLOB reads wait for writers, and get read-after-write on any replica.

docs: rondb_concurrency_control.md (Locking Unique Indexes, Locking BLOBs)

PART 1

The commit protocol

One protocol for atomicity and replication.

Two-phase commit · also the replication protocol

RonDB replicates through its commit protocol, not through a consensus log

Elsewhere

Many databases use Raft or Paxos per partition for replication, and 2PC on top for atomicity. Quorums want 3 replicas or more.

RonDB

The commit protocol writes every replica of every row it touches. No quorum: a fixed node order and an arbitrator decide after failures.

So

Highly available with two replicas. Module 15 covers arbitration.

An in-memory database can also avoid blocking: a failed node's uncommitted work is simply gone, so the survivors decide without it.

No log and no backup coordinator: transaction state lives in memory in every data node, rebuilt by a takeover after a failure. So RonDB can run millions of concurrent transactions.

docs: rondb_nonblocking_2pc.md (Atomicity vs. Replication, Blocking Protocols, Takeover Protocol)

Design · messages and waits

Two-phase commit variants: RonDB mixes the original and the linear protocol

ProtocolMessagesMessages to wait for2 rows × 2 replicas
Original 2PC: TC talks to every replica4 × rows × replicas416 · wait 4
Linear: one chain through all replicas2 × rows × replicas2 × rows × replicas8 · wait 8
RonDB: rows in parallel, a chain per row2 × (replicas + 1) × rows2 × (replicas + 1)12 · wait 6

Rows in parallel keeps latency predictable however many rows; a chain within a row saves messages and allows primary copy locking: the primary is always locked first, so replicas of one row never deadlock.

docs: rondb_nonblocking_2pc.md (Original 2PC, Linear 2PC, RonDB's 2PC; formulas as given there)

Signals · one row, two replicas, Read Backup on

A committed write, signal by signal

PhaseSignalFrom → toWhat happens
prepareTCKEYREQAPI → TCOperation, with start and commit bits
prepareLQHKEYREQTC → primary → backupEach locks the row and writes its copy
prepareLQHKEYCONFbackup → TCAll replicas prepared
commitCOMMITTC → backup → primaryGCI from DBDIH; backups record the commit, the primary installs the row and releases its lock
commitCOMMITTEDprimary → TCWithout Read Backup, TC answers the API now
completeCOMPLETE · COMPLETEDTC → backup → primary → TCBackups install and unlock; then TCKEYCONF to the API
afterTC_COMMIT_ACKAPI → TCREMOVE_MARKER_ORD clears the commit markers, which let the API learn the commit decision in case the TC fails

docs: rondb_nonblocking_2pc.md (Two vs. Three Phase Commit, RonDB's 2PC) · code: dbtc/DbtcMain.cpp (TR_DELAY_COMMIT, TF_LATE_COMMIT "wait sending apiCommit until complete phase done") · dblqh/DblqhMain.cpp (commitReqLab: the primary commits and unlocks at COMMIT, with or without Read Backup)

Signals · key reads

Key reads: the row goes to the API straight from the ldm thread or a query worker

Read committed

  • TCKEYREQ → TC → LQHKEYREQ to any replica (Read Backup).
  • The ldm thread or a query worker sends the row to the API in TRANSID_AI, directly.
  • No reply through TC: two hops less.

Locked read

  • Always to the primary replica.
  • The row goes directly too, from the ldm thread or a query worker, in TRANSID_AI; LQHKEYCONF to TC, which sends TCKEYCONF.
  • TCKEYCONF carries no data; it confirms one or more operations, even mid-transaction.

docs: rondb_signal_arch.md (Basic read transaction) · code: dbtc/DbtcMain.cpp (V_QUERY for reads without disk columns; c_use_query_thread_for_locked_reads, on by default) · module 11 covers query workers

Global checkpoints · every commit gets a GCI

Each commit is stamped with a global checkpoint id, the unit of durability

At commit

DBTC takes the current GCI from DBDIH and sends it with COMMIT. It is stored in the row header (module 07).

A pause

When the GCI moves on, every TC holds new commits for a moment, so each GCI is a clean cut through all transactions.

Durability

The REDO log is flushed per GCI. After a cluster crash, RonDB restarts from the last complete GCI: transaction-consistent.

A commit is durable at once in the memory of several nodes (network durability) and on disk at the next GCI. Module 14 covers the details.

docs: rondb_nonblocking_2pc.md (Writing and Persisting of Data) · rondb_signal_arch.md

PART 2

Failures and deadlocks

Nothing blocks for long.

TC takeover · when the coordinator fails

TC takeover: a surviving TC rebuilds the lost transactions from the ldm threads

TC on node 3failed takeover TCfirst TC on the master node LQH · node 1 LQH · node 2 LQH · node 4 … every LQH sends LQH_TRANSREQ to every LQH LQH_TRANSCONF: its operationsof node 3's transactions any node heard commit:commit · else abort

Coordinators can fail mid-transaction. A backup coordinator would not survive two failures, so RonDB rebuilds the state instead.

The first TC on the master node asks every LQH in the cluster for the operations of transactions the failed node coordinated.

Each LQH reports back in LQH_TRANSCONF. Any trace of a commit decision means commit; otherwise abort. Then the API nodes hear the outcome, and nothing stays blocked.

docs: rondb_nonblocking_2pc.md (Takeover Protocol)

Commit markers · the API always learns the outcome

Commit markers: data nodes remember a commit until the API confirms it heard

  1. A writing transaction leaves a commit marker in TC and in the LQHs it touched.
  2. If the TC node fails before the API hears the result, the takeover TC finds the markers and tells the API.
  3. With Read Backup, the final complete message rides on the "node failure handled" report: if no abort came, it committed.
  4. When the API gets TCKEYCONF it sends TC_COMMIT_ACK; TC sends REMOVE_MARKER_ORD and the markers go.

docs: rondb_nonblocking_2pc.md (Takeover Protocol) · rondb_signal_arch.md (Basic write transaction) · code: DbtcMain.cpp (commit ack marker pools, REMOVE_MARKERS)

Deadlocks · the default handling

Deadlock timeout: a transaction stuck for 1.2 s is aborted with error 266 (296 in a scan), retryable

config.iniDefaultWhat it bounds
TransactionDeadlockDetectionTimeout1200 msHow long a data node may take on one request of a transaction: lock waits included
TransactionInactiveTimeout3 600 000 msHow long TC waits for the application's next statement
  • DBTC checks its transactions periodically and adds random jitter to each timeout, so of two deadlocked transactions usually only one dies.
  • The abort carries error 266, or 296 for a scan: temporary errors, so the application retries the transaction.
  • Timeouts also catch transactions stalled by node failures.

code: ConfigInfo.cpp (TransactionDeadlockDetectionTimeout 1200, TransactionInactiveTimeout 3600000; its comment still says "around 40 days") · dbtc/DbtcMain.cpp (timeOutLoopStartLab: ndb_rand() jitter) · Dbtc.hpp (ZTIME_OUT_ERROR 266, ZSCANTIME_OUT_ERROR 296)

New in RonDB 26.10 · RONDB-1062, off by default

Proactive detection finds two transactions waiting for each other and aborts one at once

  1. When DBACC queues a lock request, it knows the waiter and the transactions blocking it: wait-for edges.
  2. Each edge goes to one collector, chosen by a hash of the transaction id, so every node agrees and no node is favoured.
  3. A collector holding both edges of a pair, A waits for B and B for A, aborts one of them at once. Longer cycles still wait for the timeout.
  4. Same errors as a timeout, 266 for key operations and 296 for scans, so applications need no change.

EnableProactiveDeadlockDetection=true in config.ini turns it on.

code: ConfigInfo.cpp (EnableProactiveDeadlockDetection, default false) · dbtc/DbtcMain.cpp (2-cycle: victim aborted with ZTIME_OUT_ERROR or ZSCANTIME_OUT_ERROR) · storage/ndb/claude_files/deadlock/design.md (RONDB-1062, phase 1: 2-cycles)

Checklist · writing transactions for RonDB

Checklist: eight habits for transactions

  1. Retry on temporary errors such as 266: they are normal.
  2. Read with FOR UPDATE when you will write the row.
  3. Lock rows in the same order in every transaction.
  4. Do not look rows up by unique key inside write transactions.
  5. Keep transactions small; watch CASCADE and large BLOBs.
  6. Lock a row that stands for a range when you need range isolation.
  7. Prefer read committed; ask for locks only where needed.
  8. No XA, no savepoints: design around them.

docs: rondb_concurrency_control.md · basic_sql.md (Transactions)

Recap · module 09

What to take away

  • Read committed readers never wait; locks are per row, strict two-phase.
  • RonDB's 2PC: rows in parallel, a chain per row, primary locked first.
  • Commit runs backup to primary; complete unlocks the backups; Read Backup answers after complete.
  • A failed TC is replaced by a takeover that commits whatever any node heard commit.
  • Deadlocks end by a 1.2 s timeout (error 266, retry), or proactively if enabled.

Next · 10 · Query execution