Key operations through an executor

The problem · work that arrives on the application's threads

Why executors: a thread per request cannot each have a connection

For programs whose work arrives on threads of their own, such as a server with a thread per request.

iclaustron: doc/user-guide.md §5 (Through an executor), §2 (one connection per thread) · crates/ic_apid/src/executor.rs (module note) · ic_port consts.rs (IC_MAX_THREAD_CONNECTIONS)

Objects · module 03 in short

The objects: one executor, a submitter per thread, shapes and requests

The executor, its shapes and its stats serve any thread; a submitter and a request belong to one thread at a time.

iclaustron: crates/ic_apid/src/executor.rs (module note; Executor, Submitter, OpShape, Request; Executor::start) · thread_conn.rs (Inbox) · apid_global.rs (ApidGlobal) · apid_conn.rs (ApidConnection)

Starting · one executor, a submitter per thread

Starting an executor: its options, then a submitter per thread

let options = ExecutorOptions {
  in_flight: 64,                     // 400 by default
  ..ExecutorOptions::default()
};
let executor = Executor::start(&global, &options)?;
let sub = executor.submitter();
let table = sub.table_bind("ic_ex", "t1")?;
// each application thread a submitter of its own:
let mine = sub.clone();
std::thread::spawn(move || { /* ... requests on mine ... */ });
// ...
drop(executor);                      // stops it, before the global

iclaustron: executor.rs (ExecutorOptions and its Default, Executor::start, submitter, stop; impl Clone for Submitter; Submitter::table_bind) · examples/rust/scan_executor.rs (run) · ic_apid tests/integration.rs (executor_with)

Shapes · the executor's form of a query

Operation shapes: a table or unique index and two records, made once

let rec = Record::default_for(&table, None)?;     // every field
let code = table.field_id("code")?;
let code_rec = Record::default_for(&table, Some(&[code]))?;

let shape = OpShape::new(&table, &rec, &rec)?;  // an Arc<OpShape>
let index = sub.index_bind("ic_ex", "t1_code$unique", "t1")?;
let by_code = OpShape::new_unique(&index, &code_rec, &rec)?;
// no free: each KeyOp holds an Arc of its shape, and the
// executor lets the shape's queries go once nobody holds it

iclaustron: executor.rs (OpShape::new, new_unique; KeyOp::new; Submitter::index_bind; Pool, Runner::query_for, sweep) · record.rs (Record::default_for) · ic_apid tests/integration.rs (pk_through; executor_runs_operations_through_a_unique_index)

Synchronous through an executor · one operation, one call

Synchronous through an executor: one operation, one blocking call

let mut req =
  Request::with_op(KeyOp::read(&shape, ReadKind::Committed));
req.ops_mut()[0].key_row_mut().copy_from_slice(&key);
sub.execute(&mut req)?;
match req.ops()[0].error() {
  Some(e) if e.code as u32 == key_op::IC_NDB_ERROR_NO_SUCH_ROW => {
    // no such row
  }
  Some(e) => return Err(e),
  None => row.copy_from_slice(req.ops()[0].attr_row()),
}

iclaustron: doc/user-guide.md §5 (Through an executor, the Rust example) · executor.rs (Request::with_op, ops_mut, KeyOp::read, key_row_mut, Submitter::execute, KeyOp::error, Request::outcome, Runner::complete_op) · key_op.rs (IC_NDB_ERROR_NO_SUCH_ROW)

Results · read once execute has returned

A request's results: each operation's outcome, then the transaction's

iclaustron: executor.rs (Submitter::execute, Request::outcome, commit_state, error, gci; KeyOp::error, error_data, result_len; Runner::finish) · transaction.rs (CommitState) · query.rs (ApidQuery::error_data: 893)

On the threads · module 02's threads, one call

A synchronous call through the threads: submit, take, send, wake

iclaustron: executor.rs (Submitter::execute, Runner::run, admit, complete_op, finish, deliver; Slot) · thread_conn.rs (submit, exchange_with_requests, post) · doc/rust/05-ndb-protocol.md (Names; §6.5)

Batched synchronous through an executor · many operations, one transaction

Batched synchronous through an executor: many operations, one call

req.clear();
for row in &rows {
  let mut op = KeyOp::write(&shape, WriteKind::Insert);
  op.key_row_mut().copy_from_slice(row);
  op.attr_row_mut().copy_from_slice(row);
  req.push(op);
}
match sub.execute(&mut req) {                  // one transaction
  Err(e) if e.code as u32 == key_op::IC_NDB_ERROR_ROW_EXISTS => {
    // rolled back: none of the rows is written
  }
  // ...
}

iclaustron: ic_apid tests/integration.rs (executor_a_request_of_several_operations_is_one_transaction: 630, none inserted) · executor.rs (Request::clear, push, set_hint; KeyOp::write, set_abort_option) · key_op.rs (IC_NDB_ERROR_ROW_EXISTS) · doc/rust/04-api-design.md (Executors)

In the objects · module 03's objects, one request of three operations

Inside the executor: each operation runs as a pooled query of its shape

iclaustron: executor.rs (Runner::admit, start_for, define_op, query_for, complete_op, finish_ended, finish; Flight) · doc/rust/04-api-design.md (Executors) · 05-ndb-protocol.md (Names)

On the threads · three callers, one pass

Batching across callers: one pass, one write per data node

iclaustron: executor.rs (Runner::run, admit_waiting; module note) · thread_conn.rs (submit: wake only if waiting) · doc/rust/architecture.html §3 (The executor's loop) · 06-phases.md §8.1 (21.0 requests a send)

Asynchronous through an executor · Rust only · beside the five models

A callback through an executor: a function gets the request back

fn note_request(request: Request, user_ref: usize) {
  // on the executor's thread, once the request has ended:
  // request.outcome(), request.ops(); user_ref is the i below
}

let mut read = KeyOp::read(&shape, ReadKind::Committed);
read.key_row_mut().copy_from_slice(&key);
let done = Done::Callback(note_request, i);
if let Err(refused) = sub.submit(Request::with_op(read), done) {
  // refused.error: 7159 busy or 7160 stopped; refused.request
}
// not waited for: the thread goes on

iclaustron: executor.rs (Submitter::submit, submit_wait; Done, RequestCallback, Refused; Runner::deliver; ExecutorShared::on_executor_thread) · auto_increment.rs (impl Submitter) · ic_apid tests/integration.rs (executor_hands_requests_to_a_callback_on_its_thread) · doc/user-guide.md §5, §9 (7159)

The loop · one pass, every pass

The executor's loop in five steps: replies before admits

1. Take, in one lock: reply pages, scan readers' nudges, and new requests while none wait in its own queue. Sleep only with nothing to do.
2. Execute the replies: operations complete, a read's row is copied into its request.
3. Hand back each request whose transaction has ended: closed, then slot and wake-up, or its callback, called there.
4. Admit waiting requests while fewer than in_flight are out: start, define, commit on the last.
5. Send what this pass admitted: one write per data node.

Replies come first: ended transactions free their coordinator records and their place under in_flight before new ones start, and results go back at once.

iclaustron: crates/ic_apid/src/executor.rs (module note; Runner::run, take_completions, deliver, admit_waiting) · doc/rust/architecture.html §3 (The executor's loop) · apid_conn.rs (poll_with_requests)

One wait point · three lists, one lock

One wait point: replies, requests and nudges share the inbox's lock

A thread can sleep on one thing only, so all three share it. A producer appends under the lock, unlocks, then wakes the executor only if it sleeps.

iclaustron: crates/ic_apid/src/thread_conn.rs (module note "Requests", "Nudges"; Inbox, post, submit, nudge, exchange_with_requests) · doc/rust/architecture.html §3 (Why both inputs must share one wait point)

Back-pressure · two bounds

Back-pressure: a bounded request list and a limit on requests out

The data nodes allow only so many transactions at once, across every API node: in_flight bounds one executor's share.

iclaustron: thread_conn.rs (submit, exchange_with_requests: room) · executor.rs (ExecutorOptions, Runner::run, admit_waiting, Submitter::submit, submit_wait, execute; Refused) · doc/rust/architecture.html §3 (Back-pressure in two places)

Failure · nothing is lost silently

Failures: every request comes back, sent or not, with its error

A temporary error: run the same request again, after a pause that grows. After 4012, learn whether a write committed before running it again.

iclaustron: executor.rs (module note; Submitter::execute, Request::set_timeout_ms; Runner::admit, abandon_defined, refuse_waiting, finish) · ic_port err.rs (IcError::is_temporary) · ic_apid tests/integration.rs (executor_deadlines_and_stopping_fail_requests_unsent) · doc/user-guide.md §9, §10

Measured · Mac, two release data nodes on the same machine, t9 by primary key

Measured: one executor reads 36 % more than a connection per thread

Your own figure: executor.stats(), then admitted over sends is requests a send.

iclaustron: doc/rust/06-phases.md §8.1 (2026-10-15; ic_bench, 100 000 keys, 10 s) · 12-performance-model.md (Calibration: M5 Pro) · user-guide.md §5 · executor.rs (Executor::stats, ExecutorStats)

The limit · what batching cannot share

What batching cannot share: a sleep and a wake-up for every call

More executors help while there are cores: two with 256 callers gave 15 % more, not twice. One per core for RonDB work, picked by key hash or round robin.

iclaustron: doc/rust/06-phases.md §8.1 (One executor is bound by its own thread; More executors) · architecture.html §3 (How many executors; What the hand-off costs) · user-guide.md §11

Also · and not yet

AUTO_INCREMENT through a submitter, and what executors cannot do yet

let id = sub.auto_increment(&table)?;

iclaustron: auto_increment.rs (impl Submitter: auto_increment and _block, _seen, _peek) · doc/user-guide.md §7, §12 · 04-api-design.md (Executors; Taking scanned rows over; Kept for later) · executor.rs (Done)

Choosing · modules 04, 06 and 05 side by side

Three ways to call: a connection per thread, an executor, asynchronous

iclaustron: doc/rust/architecture.html §4 (the table of forms A, B and asynchronous) · 06-phases.md §8.1 (measured, the Mac, 2026-10-15) · 12-performance-model.md (Calibration)

Recap · module 06

Recap: key operations through an executor

• An executor: one library thread, one connection, any number of callers.
• A shape is made once; a request is one transaction that owns its rows.
• sub.execute blocks until the transaction ends: Ok if committed.
• Or submit with Done::Callback: a function on the executor's thread gets the request, and must not block.
• Each pass: replies, then admits up to in_flight, then one write per data node.
• Every request comes back with its error; run a temporary one again.
• Writes are shared, wake-ups are not: about 10 µs a call (module 05 goes further).

Next · 07 · Scans

iClaustron training for Rust developers · 9 modules · you are at 06 · this module for C developers

Next · 07 · Scans in five models