AutomaticThreadConfig · on by default

Automatic thread configuration: the data node counts its CPUs and sets its threads

1. Find the CPUs the process may use: all of them, or those allowed by taskset or numactl.
2. Read the hardware: cores, sockets, which CPUs share an L3 cache.
3. Pick the number of ldm, tc, recv, send, main and rep threads from the CPU count.
4. Lock each thread to a CPU, grouping threads by L3 cache into round-robin groups.

For almost everyone: leave AutomaticThreadConfig=1 and set nothing else.

docs: rondb_vm_arch.md (Automatic Thread Configuration) · code: ConfigInfo.cpp (AutomaticThreadConfig default true) · thr_config.cpp

Thread counts · straight from the code's table

Thread counts: half the CPUs become ldm threads, a quarter tc threads

Up to 7 CPUs, recv threads run all the blocks (with one send thread at 6–7); from 8 CPUs the pipeline starts.

code: src/common/mgmcommon/thr_config.cpp (the table of thread counts for 1 to 48 CPUs; rows picked)

Large nodes · a formula instead of a table

Large nodes, above 48 CPUs: ldm n/2, tc n/4, recv n/8, send n/12

• CPUs left over by the divisions go first to send, then ldm, recv and tc.
• The ldm count is kept even, so round-robin groups stay balanced; an odd one becomes a send thread.
• Large nodes get more send threads because a send thread's mutex becomes a hot spot.

code: thr_config.cpp (cpu_cnt > 48: ldm cpu_cnt/2, tc /4, recv /8, send /12, remainder loop, even ldm) · 64 and 96 worked out from it

Checking · what the node decided

The node log says which threads it chose

14 CPUs: 6 ldm, 4 tc, 2 recv, 1 send, 1 main. 13 query workers: every thread except the send thread serves reads.

docs: automatic_memory_config.md (node log of a data node with NumCPUs = 14; timestamps removed, lines joined) · code: thr_config.cpp (14–15 CPUs: 6 ldm, 4 tc, 2 recv, 1 send, 1 main)

NumCPUs · for shared machines

NumCPUs fixes the thread count and turns CPU locking off

To give a data node some CPUs of a big machine, start it under numactl or taskset, give it a VM of its own, or use static CPU scheduling in Kubernetes (the static CPU manager policy): it configures and locks within those CPUs.

docs: rondb_vm_arch.md (Automatic Thread Configuration) · code: thr_config.cpp (CPU locking only when CPU information is available and NumCPUs = 0) · ConfigInfo.cpp (NumCPUs default 0, max 800)

CPU locking · 16 CPUs, 8 hyperthreaded cores

CPU locking: two ldm threads share a core; the other threads pair up on the next

code: thr_config.cpp ("We will assign 2 LDM threads, next 2 non-LDM threads"; io, idxbld and watchdog not bound; ExclusiveIoCPUs capped at 25%) · ConfigInfo.cpp (ExclusiveIoCPUs max 8)

Round-robin groups · one per L3 cache

Round-robin groups: work is shared among threads that share a cache

Data and signals stay in one cache domain: fewer cache misses than spreading over the whole machine.

code: ConfigInfo.cpp (MaxRRGroupSize 8, 8–32; UseTcInRRGroup true: "Each recv thread will distribute connections only to tc threads in the same RR group") · thr_config.cpp

CPU spinning · waiting without sleeping

CPU spinning: waking a sleeping thread can take 25 µs; spinning avoids it

Spinning suits a data node with its own machine or VM. On a shared machine it takes CPU from other programs.

docs: automated_cpu_spinning.md

SpinMethod · four levels

SpinMethod: from no spinning to spinning like a dedicated database machine

ndbinfo cpustat shows how much each thread spins; ndb_top shows it live (module 19).

code: src/kernel/blocks/thrman.cpp (spin methods, max spintime 100, 200 and 500 µs) · ConfigInfo.cpp (SpinMethod default LatencyOptimisedSpinning) · docs: automated_cpu_spinning.md

SchedulerResponsiveness · 0 to 10

SchedulerResponsiveness: higher sends sooner, lower batches more

code: ConfigInfo.cpp (SchedulerResponsiveness default 9, 0–10: "high means very responsive, low means throughput-optimised") · mt.cpp (calculate_max_signals_parameters: flush and send after 82 send wake-ups at 0, 40 at 10)

Send assistance · by load

Send assistance: block threads help the send threads until they get busy

Load is measured over 50 ms, 1 s and minutes. Separately, when the send threads pass 75% load, an idle main or rep thread is woken to help them; at 85%, two. recv threads do not help while there are ldm threads; above 32 threads, ldm and tc get nosend: threads of those types give no send assistance.

code: thrman.cpp (get_load_status: own load minus send load against 30, 60, 75 and 85, with the send threads' load deciding 75–85; check_send_thread_helpers: SEND_LEVEL_REQUIRE_ASSISTANCE 75 and EXTRA 85, off below 60 and 70; get_idle_block_threads: main and rep only) · mt.cpp (do_send; assist_send_thread: at high load only if the send thread is awake with more than 2 nodes ready; check_pending_send) · thr_config.cpp (nosend for ldm and tc above 32 threads)

ThreadConfig · every thread by hand

ThreadConfig lists each thread type with its count, CPUs and options

# config.ini, 16 cores, 32 CPUs
ThreadConfig=
 ldm={count=8,cpubind=8-15,thread_prio=10,spintime=500},
 tc={count=4,cpuset=6-7,22-23,spintime=100,thread_prio=10},
 send={count=3,cpubind=4-5,21},
 recv={count=3,cpubind=3,19,20},
 main={count=1,cpuset=0-2,16-18},
 rep={count=1,cpuset=0-2,16-18},
 io={count=1,cpuset=0-2,16-18}

ThreadConfig is used only with AutomaticThreadConfig=0. There is no query= type: RonDB 26.10 rejects it.

docs: rondb_linux.md (ThreadConfig example) · rondb_advanced_thread.md (thread_prio, realtime) · code: thr_config.cpp (thread types and parameters) · Configuration.cpp (ThreadConfig read only when AutomaticThreadConfig = 0)

Operating system · priorities and memory

OS settings: real-time priority, nice levels and locked memory are opt-in

On Linux also steer network interrupts (RSS, RPS, RFS) to the CPUs of the recv threads, and keep other work off the data node's CPUs.

docs: rondb_advanced_thread.md (thread_prio, realtime settings) · rondb_linux.md · code: ConfigInfo.cpp (RealtimeScheduler false, LockPagesInMainMemory 0, 0–2)

Small nodes · when a pipeline does not fit

Small nodes: up to 7 CPUs, recv threads do all the work

Measured on bigger nodes, dedicated ldm threads win (module 11); on very small ones balanced CPU use matters more.

code: thr_config.cpp (table rows for 1–7 CPUs) · ConfigInfo.cpp (UseLdmThreads "not using it implies only recv threads", UseTcThreads) · docs: rondb_vm_arch.md (Alternative setups)

Recap · module 12

What to take away

• AutomaticThreadConfig: half the CPUs ldm, a quarter tc, the rest recv, send, main, rep.
• Threads are locked to CPUs, two ldm per core, grouped by L3 cache.
• NumCPUs is for shared machines: fixed count, no locking.
• Adaptive spinning hides wake-up latency; LatencyOptimisedSpinning by default.
• Busy threads hand sending to send threads; idle ones help.

Next · 13 · config.ini, my.cnf and automatic memory

RonDB training · 25 modules · you are at 12

Next · 13 · Config and memory