RonDB training · module 13 of 25 · 16 min

config.ini, my.cnf and automatic memory

The two configuration files of a cluster, and how a data node divides its memory without being told.

Configuration · two files, both in ini format

config.ini describes the cluster; my.cnf describes each MySQL server

config.ini

Required. Read by the management server, which hands it to every node.
Section names and parameters are case-insensitive.
Node slots, data node settings, connections.

my.cnf

Optional: command-line options do the same.
Case-sensitive.
How a MySQL server connects to the cluster.

docs: rondb_conf_files.md (Configuration Basics)

config.ini · sections, node ids, hostnames

config.ini sections: one per node slot, with defaults for each node type

Section	Same as	Node ids
[ndbd]	[db]	1–144
[ndb_mgmd]	[mgm]	65–66 suggested
[mysqld]	[api]	67 and up suggested
[tcp] · [shm] · [rdma]	—	connections
[ndbd default] …	—	defaults for a type

Node ids go up to 8191; data nodes only 1 to 144.
Hostname is required for management servers and data nodes; for API slots it acts as a firewall.
Prefer IP addresses: RonDB aims to be more available than DNS.
Hostname is also the address the service binds to.

docs: rondb_conf_files.md (Cluster Configuration File, Node IDs, Hostnames) · rondb_rdma.md (Configuration) · code: ConfigInfo.cpp (m_sectionNames: TCP, SHM, RDMA; NodeId: data nodes 1–144, API and management nodes up to 8191)

Example · two replicas today, three tomorrow

Planning for growth: configure slots for later nodes and mark them inactive

[ndbd default]
NoOfReplicas=3
DataDir=/usr/local/ndb_data
ServerPort=11860

[ndb_mgmd]
NodeId=65
Hostname=192.168.1.100
[ndb_mgmd]
NodeId=66
Hostname=192.168.1.100
NodeActive=0

[ndbd]
NodeId=1
Hostname=192.168.1.101
[ndbd]
NodeId=2
Hostname=192.168.1.102
[ndbd]
NodeId=3
Hostname=192.168.1.102
NodeActive=0

# + [mysqld] 51, 52, [api] 231

NoOfReplicas=3, but node 3 is inactive: the cluster runs with 2 replicas.
Activate node 3 later, online, to get the third replica (module 18).
A second management server slot waits the same way.
Inactive nodes still count for PartitionsPerNode.

docs: rondb_config.md (the configuration file, shortened) · table_options.md (inactive nodes count for partitions)

Network · four ports for the firewall

Firewall ports: set ServerPort on data nodes, or they get dynamic ports

11860 is no IANA standard: RonDB recommends it because it is easy to remember next to 1186. The REST server's ports come in module 21.

docs: rondb_advanced_config.md (ServerPort, port numbers)

Disk · where files go, how big the REDO log is

Disk files: set DataDir; the default 64 GB REDO log fits almost any setup

Default REDO log

4 log parts × 16 files × 1 GB = 64 GB.
If checkpoints fall behind, RonDB speeds them up to avoid running out.
Only extreme setups need more.

A small test node

[ndbd default]
NoOfFragmentLogParts=4
NoOfFragmentLogFiles=8
FragmentLogFileSize=64M
# 4 × 8 × 64 MB = 2 GB

DataDir is enough to start; FileSystemPath and its relatives split REDO, checkpoints and disk data over devices (module 04).

docs: rondb_config.md (64 GByte of REDO log) · code: ConfigInfo.cpp (NoOfFragmentLogParts 4 to 32, NoOfFragmentLogFiles at least 3, FragmentLogFileSize 4M to 1G)

my.cnf · a MySQL server in the cluster

my.cnf: turn on ndbcluster, point at the management server, pool connections

[mysqld]
ndbcluster
ndb-connectstring=192.168.1.100:1186
ndb-cluster-connection-pool=2
ndb-cluster-connection-pool-nodeids=67,68

Each cluster connection is one API node: it takes a node id from config.ini.
One connection scales to 8–16 CPUs of the MySQL server; 4 connections reached 32 CPUs.
Fix the node ids, so logs and single user mode name the right server.

docs: rondb_advanced_mysql_config.md (--ndb-cluster-connection-pool, --ndb-nodeid) · the file is an illustration

PART 1

Automatic memory

From the machine's memory to DataMemory, with nothing set.

AutomaticMemoryConfig · on by default

Automatic memory: a data node uses the whole machine, minus what the OS needs

How much

TotalMemoryConfig = 0 reads the size from the OS, and in containers from the cgroup limit. Set it to use less: 2 GB at least.

At startup

Everything is allocated once. Every start logs the size of each region.

Scale

Production nodes mostly have 128 GB or more; tested beyond 5 TB, designed for 16 TB.

To grow, move the data node to a VM with more memory and restart it: the new size is computed automatically.

docs: automatic_memory_config.md (Memory configuration parameters) · code: ConfigInfo.cpp (TotalMemoryConfig, minimum 2G) · NdbHW.cpp (cgroup memory limit)

The calculation · everything else first, data last

Memory calculation: each region sized from the thread count; DataMemory gets the rest

Region	Default size
Left to the OS	1400 MB + 100 MB per thread, + 1.5% of memory
TransactionMemory	300 MB + 45 MB per thread
SharedGlobalMemory	700 MB + 50 MB per thread, + shares of other regions
LongMessageBuffer	32 MB + 12 MB per thread
REDO buffers · UNDO buffer	32 MB per ldm thread, at most 64 MB per log part · 24 MB per ldm thread
ReplicationMemory	8 MB per ldm thread
Job, send, schema, backup, restore …	by threads, nodes and table counts
The rest	90% DataMemory, 10% DiskPageBufferMemory; at least 512 MB

docs: automatic_memory_config.md (TransactionMemory, LongMessageBuffer, SharedGlobalMemory, ReplicationMemory, DataMemory and DiskPageBufferMemory) · code: ConfigInfo.cpp (OsStaticOverhead 1400M, OsCpuOverhead 100M) · Configuration.cpp (compute_os_overhead: memory / 67; get_and_set_redo_buffer: 32 MB per ldm, at most 64 MB per log part; get_and_set_undo_buffer: 24 MB per ldm)

Example · a real node log, 128 GB, 14 CPUs

A 128 GB node: about 99 GB became DataMemory

ndb_1_out.log real output · RonDB 24.10.8

Reserved TransactionMemory is 885 MBytes

Reserved and fixed Redo log buffer size total are 192 MBytes

Reserved and fixed LongMessageBuffer is 176 MBytes

Computed OS overhead is 13453 MBytes

SharedGlobalMemory is 1679 MBytes

Total memory is 128217 MBytes

Used memory is 18040 MBytes

Remaining memory is 110176 MBytes

Setting DataMemory to 99158 MBytes

Setting DiskPageBufferMemory to 10628 MBytes

The OS share is large here: the test set OsStaticOverhead=10G to leave room for other programs.
Everything but the data took about 4.6 GB.
Remaining memory split 90/10, minus page entry objects.

docs: automatic_memory_config.md (Example: 128 GByte workstation, NumCPUs = 14, OsStaticOverhead 10G; lines picked, prefixes removed)

Memory regions · 12 resource groups and a shared pool

Memory regions: each has a reserved part; most can borrow from a shared pool

DataMemory and the log buffers are sized once and never shared: a node must always be able to recover its data.

Other regions borrow from SharedGlobalMemory when their reserve runs out. As it fills, low priority users are refused first: query memory at 12% free, schema at 8%, transactions at 6%, replication at 4%; the rest is for job and send buffers.

code: src/kernel/vm/ndbd_malloc_impl.hpp (ULTRA 4, VERY_HIGH 6, HIGH 8, LOW 12 percent free) · src/kernel/ndbd.cpp (priority of each region) · docs: automatic_memory_config.md (Memory manager at RonDB startup)

DataMemory · the one to watch

DataMemory fills to 95% in normal use; the last 5% is for recovery

95%

the most normal operation may fill; beyond it new data is refused

100%

available during recovery, which can grow data a bit, and for reorganising

same

DataMemory on every data node: replicas need room everywhere

Shrinking DataMemory can stop a node from restarting if the recovered data no longer fits. Watch ndbinfo.memoryusage (module 19).

docs: automatic_memory_config.md (DataMemory)

Small setups · laptops, tests, containers

Small setups: NumCPUs and TotalMemoryConfig make a data node small

[ndbd default]
NoOfReplicas=2
DataDir=/usr/local/rondb_data
# Restricting CPU usage
NumCPUs=1
# Minimising memory allocation
TotalMemoryConfig=3G
RedoBuffer=16M
ReplicationMemory=50M
SchemaMemory=200M
TransactionMemory=150M
SharedGlobalMemory=150M

From the local quickstart: two data nodes on one laptop.
NumCPUs=1: one thread, no CPU locking, no spinning hogging the laptop.
Fixed small regions leave room for DataMemory in 3 GB.
The test suite runs nodes in a few hundred MB, without automatic memory.

docs: rondb_quickstart_local.md (config.ini) · automatic_memory_config.md (Examples of when to use configuration variables)

Recap · module 13

What to take away

config.ini for the cluster (case-insensitive), my.cnf for MySQL servers.
Data nodes 1–144; slots with NodeActive=0 let the cluster grow online.
Ports 1186, 11860, 3306; the default 64 GB REDO log rarely needs changing.
Automatic memory: everything sized from threads, the rest 90% DataMemory.
Shared memory is lent by priority; DataMemory fills to 95%.

Next · 14 · Durability: REDO log and checkpoints

RonDB training · 25 modules · you are at 13