RonDB training · module 03 of 25 · 18 min

Creating tables

What one CREATE TABLE statement builds inside the data nodes: keys, indexes, hidden tables and a schema transaction.

Getting started · a normal CREATE TABLE in MySQL

ENGINE=NDB makes a table a RonDB table

CREATE TABLE t1 (a INT) ENGINE=NDB;
-- NDBCLUSTER is a synonym; both are
-- case insensitive
  • Same SQL syntax as any MySQL table, same data types and character sets.
  • Create tables through a MySQL server: it stores the table definition MySQL servers need, inside RonDB.
  • From then on, every API can read and write the table: SQL, NDB API, ClusterJ, REST.

docs: basic_sql.md (Simplest form of table) · code: ha_ndbcluster.cc (SDI as extra metadata)

Primary keys · not optional in RonDB

A table without a primary key gets a hidden one

CREATE TABLE t1 (a INT) ENGINE=NDB;

-- what the data nodes store:
--   a     INT
--   $PK   BIGINT UNSIGNED, auto-increment,
--         hidden from SQL
  • Every RonDB table needs a primary key: it decides the partition and is the main access path.
  • Without one, the MySQL server adds an 8-byte hidden key column named $PK.
  • A hidden key costs 8 bytes per row and gives the application no way to look a row up by key: define your own.

docs: basic_sql.md · code: storage/ndb/plugin/ha_ndbcluster.cc (the $PK column)

Indexes · what a plain PRIMARY KEY creates

A primary key gives a distributed hash index and an ordered index

CREATE TABLE t1 (
  a INT PRIMARY KEY
) ENGINE=NDB;
hash indexDistributed: finds the partition, then the row. Used by every key lookup.
ordered indexA T-tree in every fragment replica, for range scans on the key.

Why both? Almost everyone expects an index to be ordered. The base access method is the hash index, so RonDB adds an ordered index to match that expectation. Primary key columns can never be NULL.

docs: basic_sql.md (Simple table with primary key)

USING HASH · only what you need

USING HASH drops the ordered index

CREATE TABLE t1 (
  a INT,
  PRIMARY KEY (a) USING HASH
) ENGINE=NDB;
hash indexKey lookups and full table scans work as before.
no ordered indexAbout 10 bytes per row saved, and less work on every write.

Use it when the table is only read by key. Keep the default when the application scans ranges of the primary key.

docs: basic_sql.md (Simple table with only primary key hash index, Index sizes)

Secondary indexes · KEY and INDEX

Secondary indexes: a normal index is an ordered index, local to each fragment

CREATE TABLE t1 (
  a INT, b INT,
  PRIMARY KEY (a) USING HASH,
  KEY (b)
) ENGINE=NDB;
hash index on aLookups by a.
ordered index on bRange and equality scans on b. No hash index is involved.

Each fragment has its own tree, so a scan on b visits every partition unless the query also fixes the partition key (module 04).

docs: basic_sql.md (Simple table with primary key hash index and an index, Indexes)

Unique indexes · a hidden table, kept in sync by triggers

Unique indexes: a unique key is a table of its own

create table t1 (a int, b int, primary key (a) using hash, unique key (b) using hash) engine=ndb; t1 a · primary key, hash indexb b$unique · hidden table b · primary key, hash indexa · the row's primary key in t1 Insert in t1 → insert in b$unique Delete in t1 → delete in b$unique Update of b → insert new, delete old

The unique key is enforced through a hidden index table.

It has the unique column as primary key and the base table's primary key as data: a lookup by b becomes two key lookups.

Internal triggers keep it in sync, in the same transaction as the change to t1. Without USING HASH, b also gets an ordered index.

docs: basic_sql.md (Simple table with primary key hash index and a unique index) · code: ha_ndbcluster.cc ("$unique" suffix)

ndb_desc · what the data nodes hold for a table

ndb_desc: a table seen from the data nodes' side

ndb_desc real output · mysql-test/suite/ndb
ndb_desc -a -x -dtest t1
Name: t1 Fragment type: HashMapPartition
Number of attributes: 2 Number of primary keys: 1
Row Checksum: 1 Row GCI: 1 ForceVarPart: 1
PartitionCount: 8 FragmentCount: 8 PartitionBalance: FOR_RP_BY_LDM
Table options: readbackup HashMap: DEFAULT-HASHMAP-3840-8
-- Attributes --
a Int PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY AUTO_INCR
b Int NULL AT=FIXED ST=MEMORY
-- Indexes --
PRIMARY KEY(a) - UniqueHashIndex
PRIMARY(a) - OrderedIndex
  • 8 partitions, one fragment each: PartitionsPerNode × nodes.
  • readbackup: reads may use any replica (module 02).
  • The hashmap maps 3840 buckets onto the 8 partitions.
  • One primary key, two indexes: hash and ordered.

code: mysql-test/suite/ndb/r/ndb_desc_extra.result (create table t1 (a int auto_increment primary key, b int) engine=ndb) · lines merged, values unchanged · the test cluster runs PartitionsPerNode=4, hence 8 partitions on 2 nodes

AUTO_INCREMENT · many writers at once

Auto-increment values come in ranges, one range per MySQL server

CREATE TABLE t1 (
  a BIGINT AUTO_INCREMENT,
  PRIMARY KEY (a) USING HASH
) ENGINE=NDB;

-- my.cnf, per MySQL server
ndb-autoincrement-prefetch-sz = 512
  • The counter is one row in a system table; each server fetches a range of values at a time.
  • Default 512 values, at most 65535. Small ranges make the counter row a bottleneck above about a thousand inserts per second.
  • Values are unique but not consecutive across servers.
  • Use BIGINT, so the keys never run out.

docs: basic_sql.md (A simple table with a generated primary key) · code: ha_ndbcluster.cc (autoincrement_prefetch_sz: default 512, 1–65535)

PART 1

Sizing a table

What an index costs, and the limits of a row.

Index memory · independent of the indexed columns

Index memory: an index stores row addresses, never key values

≈15 B

per row in the hash index: an 8-byte element plus container overhead

≈10 B

per row in an ordered index, mostly the row reference

0 B

more for a wider key: a 4-column index costs the same as a 1-column one

The primary key costs 15 B in its hash index plus 10 B in its default ordered index; one ordered secondary index adds 10 B: about 35 bytes of index memory per row, before the row itself. Module 08 opens the index structures.

docs: basic_sql.md (Index sizes)

Row memory · what a row costs besides its values and its indexes

Row overhead: about 34 bytes per row in a typical table, before any column value

OverheadBytes per rowWhen
Row header16Every table: operation, header bits, checksum, GCI
NULL bits4 per 32 columnsNullable columns
Variable part reference8Tables with variable-size columns, which is almost every table. MySQL reserves it in the rest too, unless ROW_FORMAT=FIXED
Variable part4 + 2Tables with variable-size columns: page directory entry and end offset
Variable-size column2 eachVARCHAR, VARBINARY, TEXT, BLOB: the column's offset
Disk part reference8Tables with disk columns

Typical table: 16 + 4 + 8 + 4 + 2 = 34 bytes, plus 2 per VARCHAR, padded to whole words. Add the index memory, then multiply by NoOfReplicas.

code: Dbtup.hpp (Tuple_header, Var_part_ref) · tuppage.hpp · DbtupExecQuery.cpp (shrink_tuple) · DbtupMeta.cpp · ha_ndbcluster.cc (ForceVarPart) · layout in module 07

Limits · in RonDB 26.10

Table limits: how big a table and a row can be

LimitRonDB 26.10
Columns per table4096
Columns per index or key32
Indexes per table64
Row, all columns72 000 bytes
Fixed-size part in memory16 280 bytes
Variable-size part in memory32 632 bytes
Disk part32 632 bytes
Primary key, through the NDB API4092 bytes
Any key, through a MySQL server3072 bytes

code: ndb_limits.h (MAX_ATTRIBUTES_IN_*, MAX_*_SIZE_IN_WORDS; MAX_VAR_SIZE_IN_WORDS and MAX_DISK_VAR_SIZE_IN_WORDS 8158) · ndbapi_limits.h (NDB_MAX_KEY_SIZE) · sql_const.h (MAX_KEY_LENGTH) · handler.h (max_key_length: the smaller) · CMakeLists.txt (MAX_INDEXES) · BLOBs: module 04

Columns · any MySQL type, with a few index limits

Column types and character sets

Supported

  • Every MySQL column type, every character set.
  • JSON columns.
  • GIS columns, without GIS indexes.
  • Indexes on stored generated columns.

Not supported

  • GIS (spatial) indexes and fulltext indexes.
  • Indexes on virtual generated columns.
  • Indexes on columns stored on disk: indexed columns live in memory.

The default character set is utf8mb4: up to 4 bytes per character. Set a smaller one per table or per column when the data allows it.

docs: basic_sql.md (More complex tables, Indexes)

PART 2

One statement, many objects

A CREATE TABLE is a distributed schema transaction.

Schema objects · created together, or not at all

Schema objects: what one CREATE TABLE creates in the data nodes

The tableIts partitions as fragments on every node of every node group.
A hashmapDEFAULT-HASHMAP-3840-NBuckets to partitions.
Hash indexOn the primary key.
Ordered indexesOne per KEY, plus PRIMARY unless USING HASH.
Unique index tables<index>$uniqueWith their triggers.
BLOB part tablesNDB$BLOB_<table id>_<column>One per BLOB or TEXT column.
Foreign keysChecked in the data nodes, for every API.
The table definitionFor the MySQL servers, stored in RonDB.

docs: rondb_schema_transaction.md (Transaction Steps) · foreign_keys.md · code: ha_ndbcluster.cc · storage/ndb/src/ndbapi/NdbBlob.cpp (NDB$BLOB names)

Schema transactions · slow, but very reliable

Schema transactions: a change commits or rolls back as a unit, through node failures

mysqldGlobal schema lock master data nodeDBDICT data node · DBDICT data node · DBDICT Before commit pointroll back Commit point After commit pointroll forward Every data node logs completed phases in its schema log · a restarting node finishes or aborts the transaction from it

The statement is split into schema operations; the DBDICT block runs each through 8 phases, every phase with its own rollback.

A failure before the commit point rolls all finished phases back. After it, the surviving or restarted nodes finish the transaction.

While it runs: no other schema transaction, no backup, no node start. Run schema changes when nothing else needs those.

docs: rondb_schema_transaction.md · code: dbdict (the log file is P0.SchemaLog)

Schema distribution · every MySQL server learns about the table

Schema distribution: RonDB pushes the definition to every MySQL server

mysqld ARuns the CREATE TABLE data nodes t1 + its definition (SDI) row in mysql.ndb_schema Event mysqld BInstalls t1 in its dictionary mysqld CInstalls t1 in its dictionary

The MySQL server running the DDL stores the table definition (SDI) in the table's metadata in RonDB, and logs the change in mysql.ndb_schema.

An event on that table reaches every other MySQL server: each installs the table and acknowledges. API nodes read the definition straight from RonDB.

code: ha_ndbcluster.cc (SDI as extra metadata) · storage/ndb/plugin/ndb_schema_dist_table.cc (mysql.ndb_schema, event REPL$mysql/ndb_schema)

PART 3

MySQL in a cluster

Some objects live in the data nodes, others in each MySQL server.

MySQL objects · where each one lives

Tables are cluster-wide; views, triggers and procedures are per server

ObjectWhere it livesWhat to do
Databases, tables, indexes, foreign keysRonDB, cluster-wideCreate once, from any MySQL server
Views, stored procedures, functionsEach MySQL serverCreate them on every server that uses them
MySQL triggersEach MySQL serverThey do not fire for NDB API, ClusterJ or REST writes
EventsEach MySQL serverInstall on a few, make them run once per period
Users and grantsPer server, or cluster-wideGrant NDB_STORED_USER to store a user in RonDB

docs: mysql_concepts_in_ndb.md · code: storage/ndb/plugin/ndb_stored_grants.cc (NDB_STORED_USER)

Transactions · what MySQL features carry over

Every statement runs as a distributed transaction

Supported

  • Autocommit, and BEGIN · COMMIT · ROLLBACK.
  • Large inserts split into several transactions (ndb-use-transactions=1, the default); LOAD DATA always is.

Not supported

  • XA transactions.
  • Savepoints.
  • LOCK TABLES across servers: it only locks the table in one MySQL server.

docs: basic_sql.md (Writing to a table, Transactions) · mysql_concepts_in_ndb.md (Savepoints, Table locks, XA support)

Housekeeping · three statements that behave differently

COUNT(*), ANALYZE TABLE and OPTIMIZE TABLE

SELECT COUNT(*)

A quick estimate from a row counter in each fragment. ndb-use-exact-count=1 scans the whole table instead: much slower, and no other server is locked out meanwhile.

ANALYZE TABLE

Builds the index statistics the optimizer uses: large speed-ups for some queries.

OPTIMIZE TABLE

Defragments variable-size and dynamic row parts. The fixed part stays: moving it would change row ids, which recovery depends on.

docs: basic_sql.md (SELECT COUNT(*) queries, Optimize/Analyze Table)

Checklist · before a table goes into production

Checklist: eight habits for RonDB tables

  1. Define a primary key; avoid the hidden $PK.
  2. Use USING HASH when the key is never scanned by range.
  3. Make auto-increment keys BIGINT; raise the prefetch size for heavy inserts.
  4. Pick a character set per column when utf8mb4 is not needed.
  5. Remember every index costs 10–15 bytes per row.
  6. Create tables through a MySQL server.
  7. Keep MySQL triggers out of your constraints: other APIs bypass them.
  8. Run ANALYZE TABLE after loading data.

docs: basic_sql.md · mysql_concepts_in_ndb.md

Recap · module 03

What to take away

  • ENGINE=NDB, and always a primary key: otherwise a hidden $PK.
  • A primary key is a hash index plus an ordered index; USING HASH keeps only the hash.
  • Unique keys are hidden tables kept in sync by triggers.
  • Indexes store row addresses, not keys: about 15 bytes per row for hash, 10 for ordered.
  • A CREATE TABLE is a schema transaction: all objects or none, distributed to every MySQL server.

Next · 04 · Table options, partitioning, disk columns and BLOBs