Backup · not the same as a checkpoint

Backups vs checkpoints: backups survive disasters and move data

Both share one disk write budget, MinDiskWriteSpeed and MaxDiskWriteSpeed; BackupDiskWriteSpeedPct (50) reserves part of it for backups.

docs: rondb_backup.md · code: ConfigInfo.cpp (BackupDiskWriteSpeedPct 50: "Percentage of MaxDiskWriteSpeed to reserve for Backup")

Taking a backup · from the management client

START BACKUP: one command backs up the whole cluster, online

docs: rondb_backup.md · rondb_global_prepare_primary.md (the output) · code: mgmclient/CommandInterpreter.cpp (START BACKUP)

START BACKUP · the full syntax

START BACKUP options: the id, encryption, the snapshot point and waiting

START BACKUP [<backup id>] [ENCRYPT [PASSWORD='<password>']]
    [SNAPSHOTSTART | SNAPSHOTEND] [NOWAIT | WAIT STARTED | WAIT COMPLETED]

code: mgmclient/CommandInterpreter.cpp (helpTextStartBackup) · ConfigInfo.cpp (RequireEncryptedBackup) · docs: rondb_backup.md (The START BACKUP command)

The algorithm · a scan plus a log

The backup algorithm: scan every fragment while a log records the changes

docs: rondb_backup.md (Backup algorithm) · code: backup/Backup.cpp

SNAPSHOTEND or SNAPSHOTSTART · REDO or UNDO

SNAPSHOTEND or SNAPSHOTSTART: the backup shows its end or its start

docs: rondb_backup.md (Backup algorithm, The START BACKUP command) · code: backup/Backup.cpp (log left uncompressed for SNAPSHOTSTART)

On disk · one set of files per ldm thread

Backup files: every ldm thread writes its own part of the backup

# under BackupDataDir, on data node 1
BACKUP/
  BACKUP-10/
    BACKUP-10-PART-1-OF-4/
      BACKUP-10-0.1.Data   # rows
      BACKUP-10.1.ctl      # definitions
      BACKUP-10.1.log      # changes
    BACKUP-10-PART-2-OF-4/
      …
    BACKUP-10-PART-4-OF-4/

code: src/kernel/blocks/ndbfs/Filename.cpp (BACKUP-%u-PART-%u-OF-%u, BACKUP-%u-%u.%u) · backup/Backup.cpp · docs: rondb_backup.md (Backup file placement) · ids are an illustration

Configuration · and file management

Put backups on their own device: a full disk can stop the cluster

Old backups are yours to delete. ABORT BACKUP <id> stops a running backup and deletes its files.

docs: rondb_backup.md (Backup configuration, Backup file management, Aborting a backup) · code: ConfigInfo.cpp (BackupLogBufferSize 16M, CompressedBackup false, BackupDiskWriteSpeedPct 50)

Metadata · what the backup does not cover

Back up MySQL objects with mysqldump

mysqldump \
  --no-data \
  --databases db1 db2 \
  > backup_metadata.sql

# restore it through a MySQL server
mysql < backup_metadata.sql

docs: rondb_backup.md (Backing up metadata in RonDB) · rondb_restore.md (Using MySQL dump)

ndb_restore · into a freshly started cluster

ndb_restore: metadata once, indexes off, data in parallel, indexes back on

Restore into an empty cluster, after an initial start. The target may have a different number of data nodes: rows are hashed to its partitions.

docs: rondb_restore.md (Commands to run) · code: tools/restore/restore_main.cpp

Commands · backup 10 of a two-node cluster

ndb_restore commands: each names the server, node id, backup id and path

R="ndb_restore --connectstring=mgm_host --backupid=10 --backup-path=/backup/path/dir"

$R --nodeid=1 --restore_meta --no-restore-disk-objects    # 1 once
$R --nodeid=1 --disable-indexes                           # 2 once
$R --nodeid=1 --restore-data &                            # 3 per node,
$R --nodeid=2 --restore-data &                            #   in parallel
wait
$R --nodeid=1 --rebuild-indexes                           # 4 once

• --nodeid names the node that took the backup, whose files are read.
• Each call finds the PART directories itself and restores them with a thread each; --parallelism defaults to 128 transactions.
• After step 1, run SHOW TABLES on the MySQL servers so they see the new tables.

docs: rondb_restore.md (Commands to run, Restoring metadata) · code: restore_main.cpp (parallelism default 128; PART-x-OF-y detection) · the shell wrapper is an illustration

Indexes · off during the data step

Why indexes are off: restored data is inconsistent until the last log record

docs: rondb_restore.md (Disabling indexes, Commands to run)

ndb_restore options · pick and transform

ndb_restore options: restore part of a backup, under other names, or with changes

code: storage/ndb/tools/restore/restore_main.cpp (option table) · docs: rondb_restore.md (Partial restore)

Global replication · start a replica from a backup

Seeding a replica: a restore tells it where in the binary log to start

1. Back up the primary cluster; the backup ends at a known epoch (from StopGCP).
2. Restore all four steps on the replica with --no-binlog: the restore itself must not be replicated.
3. Add --restore-epoch to the last step: it writes the epoch into ndb_apply_status with server_id 0.
4. Start replication from that epoch: nothing lost, nothing applied twice (module 24).

docs: rondb_restore.md (restore with --no-binlog and --restore-epoch) · rondb_global_prepare_primary.md

Recap · module 17

What to take away

• START BACKUP: online, consistent at a global checkpoint.
• A scan plus a log: SNAPSHOTEND rolls forward, SNAPSHOTSTART rolls back.
• One PART directory per ldm thread, on every data node; use a separate device.
• Restore: metadata, indexes off, data per node in parallel, indexes on.
• --no-binlog and --restore-epoch seed global replication.

Next · 18 · Operations

RonDB training · 25 modules · you are at 17

Next · 18 · Operations