RonDB training · module 01 of 25 · 14 min

What RonDB is and why it exists

A distributed key-value database with SQL, built for predictable latency and availability that never stops.

RonDB · in one slide

RonDB: a distributed, shared-nothing database with its data in memory and on disk

  • A fork of MySQL NDB Cluster, the NDB storage engine of the MySQL server.
  • In-memory and on-disk columns, up to 1 PiB of storage.
  • Online scaling of storage and of compute, both horizontal and vertical.
  • ACID: in-memory data is made durable through a REDO log and checkpoints.
  • Open source, written in C++, developed by Hopsworks.
Active-activeActive-passive
Synchronous, across availability zones1–3 replicas inside one cluster—
Asynchronous, across regionsn clusters, with conflict resolutionn clusters

Replication inside a cluster is synchronous; between clusters it is asynchronous.

docs: index.md · config_num_replicas.md

LATS · the four properties RonDB delivers together

LATS: Low Latency, high Availability, high Throughput, scalable Storage

LLow LatencyA simple transaction can complete in less than 100 µs.
Ahigh AvailabilityReadable and writable through upgrades, restarts and most schema changes.
Thigh ThroughputMillions of reads and writes per second, scaling with the cluster.
Sscalable StorageMemory and disk columns, up to 1 PiB, scaled online.

Many databases have two or three of these. RonDB is built to deliver all four at once: the fastest key-value store with SQL support.

docs: index.md · rondb_use_cases.md

PART 1

Origins: a telecom database

Requirements from network databases at Ericsson that still shape RonDB.

History · from a PhD study to RonDB

RonDB history: three decades in production systems

1991First thoughts on NDB, during Mikael Ronström's PhD studies at Ericsson: the UMTS network database study.
1997The Ericsson NDB project starts: a Number Portability prototype for a telecom vendor.
2003NDB becomes part of MySQL: MySQL NDB Cluster.
2004In production; today at least several tens of thousands of clusters.
21.04 → 26.10RonDB, the fork developed by Hopsworks: cloud-native, LATS, the online feature store.

docs: rondb_rationale.md · rondb_use_cases.md (Number Portability Demo) · rondb_special.md · rondb_innodb.md · new_features_2104.md

The study · UMTS, number portability, DNS, intelligent networks, news-on-demand…

The PhD study: ten conclusions that still shape RonDB

  1. Predictable response time requires a main-memory database
  2. Availability requires a shared-nothing architecture
  3. Predictable response time requires a real-time scheduler
  4. Applications must not execute inside the DBMS, unless protected
  5. Throughput requires a main-memory database
  6. Some applications need large objects on disk inside the DBMS
  7. Throughput requires a scalable database
  8. The most common operation is a key lookup
  9. Availability requires replicated data
  10. Most applications are write-intensive

docs: rondb_rationale.md (introduction)

PART 2

AlwaysOn, and where RonDB runs

Less than 30 seconds of downtime per year takes more than a replication algorithm.

AlwaysOn · Class 6, less than 30 seconds of downtime a year

AlwaysOn: a clever replication algorithm is one of seven ingredients

1ReplicationSynchronous replicas inside the cluster.
2Instant failoverShared nothing: no log replay, sub-ms takeover.
3Global replicationA second cluster for power cuts, storms and big rewrites.
4Failfast softwarendbrequire stays on in production: a wrong state stops the node.
5Modular softwareBlocks and signals, after AXE.
6Crash analysisJump Address Memory and per-thread signal logs.
7Managed softwareOperated by people and processes, 24×7.

docs: how_to_always_on.md

Availability · readable and writable, all the time

Availability classes: from "high availability" to AlwaysOn

LevelAvailabilityDowntime per yearHow
Typical cloud99.95%up to 260 minutesreplication across availability zones
Class 499.99%no more than 50 minuteswhat most IT calls "high availability"
Class 599.999%no more than 5 minutesone RonDB cluster, online through upgrades and schema changes
Class 6 · AlwaysOn99.9999%less than 30 secondsRonDB clusters in two data centres, with global replication

docs: rondb_use_cases.md (introduction) · rondb_special.md

Use cases · billions of people use NDB without knowing it

Use cases: where NDB and RonDB run

TelecomHLR, number portability, call setup.
NetworkingDNS, DHCP and RADIUS servers.
Gaming and financeOnline games, trading, payments, betting.
Web and LDAPWeb applications and LDAP servers.
HopsFSFile-system metadata: 1.4M file ops/s, 60 name nodes on 12 data nodes.
Feature storeThe Hopsworks online feature store.
Fraud detectionFeatures computed at request time.
NoSQL appsKey-value through native APIs, SQL too.

docs: rondb_use_cases.md

Scale · reads and writes

RonDB scale: millions of writes per second, which very few systems reach

205M

reads per second, demonstrated by NDB

20M

writes per second, the same benchmark

104.5M

key lookups per second through the REST API, RonDB 24.10

A distributed transaction reading and writing hundreds of rows finishes in a few milliseconds; a simple transaction can complete in less than 100 µs.

docs: rondb_use_cases.md (introduction) · www.rondb.com (REST benchmark, 6 data nodes, AWS Graviton4)

Fit · an honest boundary

When RonDB is not the best fit

Hard to beat RonDB

  • High availability is required.
  • High update rates, and replication.
  • Predictable latency under mixed load.
  • Data and load larger than one machine.

Where it struggles

  • No need for availability, and everything fits in one machine: an in-memory database without context switches is hard to beat there.
  • Pure analytics: RonDB stores rows, for OLTP with some analytical queries.

docs: rondb_use_cases.md (introduction) · rondb_rationale.md (OLTP, OLAP and relational models)

PART 3

Design: a data server, separate from query servers

Why RonDB is a key-value database with SQL as an added value, and an architecture borrowed from a telephone switch.

Data server and query server · where to draw the line

Three ways to put an application, a query server and a data server together

1 · Fully integrated One binary, one memory application dbms Fastest · a stray pointerin the app corrupts data 2 · Colocated (InnoDB, Oracle, DB2) application SQL only One server process query data Every access goes throughThe one query language 3 · RonDB mysqld NDB API REST LDAP · FS data servers · data nodesKey lookups, scans, events Query servers reach the dataover a small network protocol

Fully integrated: a key lookup in AXE's DBS took 2 assembler instructions. But the application can damage the data.

Colocated is the classic design. It is specialised: if the API is SQL, every access goes through SQL.

RonDB separates them: the data server offers key lookups, scans, schema changes and event streams over a network protocol; query servers sit on top.

Disk databases now split too: storage servers apart from query servers, close to RonDB's division. But RonDB's data servers also handle in-memory data.

docs: rondb_rationale.md (Separation of Data Server and Query Server)

Trade-off · what the separation costs and what it buys

Separating query and data servers costs one network hop and buys availability and scale

The cost

  • An SQL query passes through several threads and a network hop to reach the data.
  • A combined query and data server is always faster on a single machine.
  • Colocating mysqld and the data node on one host brings the two close again.

What it buys

  • Only a small, checked API can change the data.
  • A fast path for key lookups that skips SQL entirely.
  • SQL, NDB API, LDAP and file systems on the same data.
  • Queries parallelised inside the data nodes.

For SQL, RonDB is close to MySQL/InnoDB; once its parallel query execution comes into play, it is faster.

docs: rondb_rationale.md (Drawbacks of RonDB Model, Advantages and Disadvantages)

Predictable response time · a real-time scheduler inside the data node

Predictable latency: short signals keep a heavy query from blocking a key lookup

One thread's signal queue, time → scan scan scan scan scan A complex query, split into signals thatScan a few rows, each under 10 microseconds Scan Key Scan Key Key Scan Key Scan Key lookups, under 2 microseconds each, slot in between the scan's signals

A complex query is cut into short signal executions; each puts itself back in the queue and waits for its next turn.

Traffic queries slot in between. With no traffic, the complex query gets the CPU to itself. Latency stays stable up to about 90% load.

docs: rondb_rationale.md (Predictable response time requirements, How does it work)

Shared-nothing · why every telecom database chose it

Shared-nothing: failover without replaying a log

Shared disk node 1 node 2 node 3 shared storage · global REDO log replay the global redo log before node 3's pages can be used again Shared nothing Node group 0 node 1Data+ logs node 2Data+ logs Node group 1 node 3Data+ logs node 4Data+ logs node 4 already holds the same data: only the distribution changes

Shared disk scales a few big servers over one storage system. Shared nothing gives every node its own data and logs, replicated in its node group.

Every transaction updates data and logs on all replicas, so failover only updates the distribution: under a millisecond once the failure is detected.

docs: rondb_rationale.md (Shared-nothing architecture) · how_to_always_on.md

The CAP theorem · two answers, at two levels

CAP theorem: consistent inside a cluster, available across clusters

Inside one cluster: C + A

If the network splits the cluster, both halves are not allowed to continue: that would break consistency. RonDB survives as many partition scenarios as it can without giving up consistency.

Between clusters: A + P

Replication to another region is asynchronous, in groups of transactions (epochs). If the network between them splits, every cluster keeps running; writes in all clusters are possible with conflict resolution.

docs: rondb_rationale.md (RonDB and the CAP theorem)

Heritage · the AXE telephone switch, Ericsson, 1970s

AXE heritage: blocks that own their data, signals as the only way in

Hardware chip Wire chip AXE and RonDB block DBTCowns its own data:transactions Signal block DBLQHowns its own data:fragments, operations Signal block DBTUPowns its own data:rows, pages What it buys A wrong state comes from the block's own code or from a signal sent to it. Signals are traced: the last few thousand per thread are in every crash report.

AXE's engineers copied hardware: an integrated circuit only talks to another through a signal on a wire.

So a block owns its data and nothing can peek at it; the only way in is a signal. RonDB's data node is built the same way, as large C++ classes.

The architecture makes bugs findable, and a virtual machine runs the signals with almost no cost to switch between tasks.

docs: how_to_always_on.md · rondb_rationale.md (AXE VM)

Recap · module 01

What to take away

  • RonDB is a fork of NDB Cluster: a distributed, shared-nothing database for LATS.
  • It was born from telecom requirements: main memory, a real-time scheduler, replication, shared nothing.
  • AlwaysOn, less than 30 seconds of downtime a year, takes seven ingredients, not only replication.
  • The data server is separate from query servers: SQL, NDB API, REST and more on the same data.
  • Consistent inside a cluster, available across clusters.
  • Blocks and signals, after AXE, give predictable latency and findable bugs.

Next · 02 · Cluster anatomy: nodes, node groups, partitions and replicas

RonDB training · 25 modules · you are at 01

Next · 02 · Cluster anatomy