The API map · one road into the data nodes

Every way into RonDB ends in the C++ NDB API

Learn the NDB API and you understand what every other interface does underneath. Modules 21 and 22 cover REST and RonSQL.

docs: rondb_ndb_api.md · rondb_clusterj.md · notes: apis.md (rdrs2 hosts REST, RonSQL, Rondis)

NDB API · four levels of objects

NDB API objects: a cluster connection, Ndb objects, transactions, operations

An Ndb object is used by one thread at a time; give each thread its own. init() sets how many transactions it may run at once: 4 by default, at most 1024.

docs: rondb_ndb_api.md (Concurrency in the NDB API, Ndb cluster connection) · code: include/ndbapi/Ndb.hpp (init(int maxNoOfTransactions = 4), maximum 1024)

Connecting · from the shipped examples

Connecting: reach the management server, wait for data nodes, init an Ndb

ndb_init();
Ndb_cluster_connection connection(connectstring);
if (connection.connect() != 0) /* no mgmd */;
if (connection.wait_until_ready(30, 0) != 0)
  /* no data node within 30 s */;

Ndb ndb(&connection, "ndbapi_examples");
if (ndb.init() != 0) /* error */;

code: storage/ndb/ndbapi-examples/ndbapi_basic/ndbapi_basic_connect.cpp, ndbapi_basic_insert.cpp (error handling shortened) · docs: rondb_ndb_api.md

Inside an API node · threads you do not create

API threads: each cluster connection runs receive, send and manager threads

Only one thread at a time can receive for a connection: the limit that makes very busy programs open more connections.

docs: rondb_api_arch.md (NDB API send threads, receive threads, cluster manager threads, wakeup thread) · rondb_ndb_api.md

Transactions · define, then execute

Transactions: define operations locally, then execute them in one round trip

1. startTransaction(table, key): the key is a hint that starts the transaction on the node holding that row (module 10).
2. Define operations: nothing is sent yet.
3. execute(NoCommit) sends them and keeps the transaction open; execute(Commit) sends and commits; Rollback aborts.
4. closeTransaction() always, to free its resources.

AbortOnError aborts the transaction on any failed operation; AO_IgnoreError lets others go on, for example a read of a row that may not exist.

code: include/ndbapi/NdbTransaction.hpp (ExecType NoCommit, Commit, Rollback) · NdbOperation.hpp (AbortOnError, AO_IgnoreError) · docs: rondb_ndb_api.md

Key operations · an insert, from the examples

An insert: get the table, start a transaction, define, execute, close

const NdbDictionary::Table *table =
    ndb.getDictionary()->getTable("basic");
NdbTransaction *trans = ndb.startTransaction(table);
NdbOperation *op = trans->getNdbOperation(table);

op->insertTuple();
op->equal("ATTR1", key);      // primary key
op->setValue("ATTR2", value);

if (trans->execute(NdbTransaction::Commit) != 0)
  /* trans->getNdbError() */;
ndb.closeTransaction(trans);

code: ndbapi-examples/ndbapi_basic/ndbapi_basic_insert.cpp (do_insert; error handling shortened) · include/ndbapi/NdbOperation.hpp

Batching · the biggest win

Batching: many operations per execute, one round trip, run in parallel

docs: rondb_ndb_api.md · notes: apis.md (flexAsynch; batching about 10×)

Scans · with NdbRecord, from the examples

Scans with NdbRecord: describe the row once, scan, loop over nextResult

NdbDictionary::RecordSpecification spec[] = {
  {table->getColumn("ATTR1"), offsetof(BasicRow, attr1), 0, 0, 0},
  {table->getColumn("ATTR2"), offsetof(BasicRow, attr2), 0, 0, 0}};
NdbRecord *record = dict->createRecord(table, spec, 2, sizeof(spec[0]));
NdbTransaction *trans = ndb.startTransaction(table);
NdbScanOperation *scan = trans->scanTable(record);
trans->execute(NdbTransaction::NoCommit);      // start the scan
const BasicRow *row;
while (scan->nextResult((const char **)&row, true, false) == 0)
  use(row->attr1, row->attr2);                // batches arrive as you go
scan->close();
ndb.closeTransaction(trans);

NdbRecord maps columns to a C struct once: rows land in your memory without per-column calls. scanIndex scans an ordered index with bounds.

code: ndbapi-examples/ndbapi_basic/ndbapi_basic_read.cpp (std::size replaced by 2, errors shortened, use() is a placeholder)

NdbScanFilter · conditions run in the data nodes

A scan filter becomes an interpreted program on every row

NdbScanFilter filter(scan);
filter.begin(NdbScanFilter::AND);
filter.cmp(NdbScanFilter::COND_GE,
           col_price, &min_price, 4);
filter.cmp(NdbScanFilter::COND_EQ,
           col_region, &region, 4);
filter.end();

code: include/ndbapi/NdbScanFilter.hpp (begin, cmp, end) · NdbInterpretedCode.hpp · ndbapi-examples/ndbapi_agg (NdbAggregator) · the column ids and values are illustrations

Asynchronous API · many transactions per thread

Asynchronous API: one thread keeps hundreds of transactions in flight

code: ndbapi-examples/ndbapi_async (executeAsynchPrepare, sendPollNdb) · docs: rondb_ndb_api.md

Events · from the examples

Events: subscribe to a table and receive every insert, update and delete

NdbEventOperation *op =
    ndb->createEventOperation(eventName);
NdbRecAttr *after  = op->getValue("COL0");
NdbRecAttr *before = op->getPreValue("COL0");
op->execute();                  // changes start flowing

while (running) {
  if (ndb->pollEvents(1000) > 0)   // wait ≤ 1 s
    while ((op = ndb->nextEvent()))
      switch (op->getEventType()) {
        case NdbDictionary::Event::TE_INSERT: …
        case NdbDictionary::Event::TE_UPDATE: …
        case NdbDictionary::Event::TE_DELETE: …
      }
}

code: ndbapi-examples/ndbapi_event/ndbapi_event.cpp (lines picked and shortened; column name simplified)

Events · delivered by epoch

Events by epoch: changes arrive complete and ordered, every 100 ms

docs: rondb_checkpoints_global_redo.md (micro GCPs for replication) · notes: apis.md (events per GCI, node failure buffering, users)

ClusterJ · interfaces mapped to tables

Annotate an interface; ClusterJ reads and writes rows through it

@PersistenceCapable(table="customer")
public interface Customer {
  @PrimaryKey
  @Column(name="customer_id")
  int getId();
  void setId(int id);
  String getName();
  void setName(String name);
}

Session s = sessionFactory.getSession();
Customer c = s.newInstance(Customer.class);
c.setId(100); c.setName("Mikael");
s.persist(c);                 // autocommit insert
Customer r = s.find(Customer.class, 100);

docs: rondb_clusterj.md (Customer interface, Session, persist, makePersistentAll, find, clusterj.properties)

Recap · module 20

What to take away

• Connection → Ndb (one thread at a time) → transaction → operations.
• Define many operations, execute once: batching is the biggest win.
• NdbRecord for rows, NdbScanFilter for pushed conditions, async for many transactions per thread.
• The Event API streams every change, grouped by epoch.
• ClusterJ maps Java interfaces to tables on the same API.

Next · 21 · The REST API

RonDB training · 25 modules · you are at 20

Next · 21 · REST API