Multicore

LuraDB spreads its domains over several data shards by name; each shard is one thread with its own storage engines. Reads scale across cores, while the writes of one domain stay on one.

How requests reach the cores

Every domain lives on one shard, so its writes run on one core

Reads of one domain run on many cores at once

The shard numbers in the drawings are examples. LuraDB hashes the name of a domain into one of 256 fixed slots and gives the slot to shard number slot mod shards; the result depends only on the name and the shard count, nothing is stored per domain, and it is the same in all three engines.

What that means for one domain

• Reads scale across cores. Point reads run on the thread that holds the connection, longer reads on a pool of reader threads, both against the shard of the domain, from any number of connections at once.
• Writes of one domain run on exactly one core, the thread of its shard. Write parallelism is at most the number of domains being written at the same time; spread independent write traffic over several domains.
• One connection serves one request at a time, also with HTTP/2. A client that wants parallelism opens several connections.

Each kind of thread has one job

Thread	Sized by	Job
Frontend workers	multicore.frontend_workers	Accept connections, terminate TLS, parse HTTP, find the shard of the domain in the path, answer point reads, hand everything else on.
Data shards	multicore.shards	Each shard is one thread with its own storage engines: it runs every write of the domains it owns.
Reader pool	multicore.reader_threads	Runs the longer reads of all shards: scans, listings, searches and SELECT statements.
Control plane	one	Runs the requests that name no domain, such as creating a domain or a user, and carries the backup and restore jobs.
Storage thread	one	Writes the write-ahead logs of all engines, see Write path.
CPU offload	multicore.cpu_offload_threads	Takes CPU-heavy parsing, such as the body of a bulk import, off the engine threads.

The MCP listener, when enabled, runs on a runtime of its own with one worker, so that its long-lived streams take no frontend worker.

A request waits in a queue when its thread is busy, and the queues are bounded. A data shard takes multicore.shard_queue_capacity requests at once, running or waiting; the reader pool queues multicore.reader_queue_capacity reads. A request beyond that is answered 503 Service Unavailable with Retry-After: 1: it was not processed and may be retried.

Automatic sizing

The cores are split automatically between frontend workers (HTTP/TLS), data shards and reader threads, and [multicore] in luradb.toml overrides each part.

Part	Automatic size	4 cores	8 cores	12 cores	32 cores
Frontend workers	cores / 6, 2 to 4; 1 below 4 cores	2	2	2	4
Data shards (new data directory)	cores / 4, 1 to 8	1	2	3	8
Reader threads	cores − 2 − frontend − shards, at least 1	1	2	5	18

The start log reports the split. On a host limited to four cores:

INFO luradb::server: 4 cores: 2 frontend workers, 1 data shards, 1 reader threads; shard queue capacity: 1024
INFO luradb::server: Reader pool queue capacity: 4096

The CPU offload takes the cores minus two, at least one; the log says CPU offload permits: 2 there. A shard count that you set applies as given; the table shows the automatic size for a data directory that is created now. For an existing one, see below.

Every shard adds memory

The memory budgets of the storage engines are split across the shards, except the MemTable, which is per shard. The block cache, 64 MiB per engine, and the buffer pool of buffer_pool.pool_size pages are divided evenly, so more shards do not mean more cache. The MemTable threshold lsm.memtable_size_threshold applies to every engine on every shard, so MemTable memory grows to engines × shards × threshold, plus the MemTables being flushed. The start log shows each shard’s share. A server with one shard and one with eight:

INFO luradb::server::control: Shard 0: block cache 65536 KiB per engine; MemTable threshold 4194304 bytes per engine; buffer pool 1024 pages; registered I/O buffers off.

INFO luradb::server::control: Shard 0: block cache 8192 KiB per engine; MemTable threshold 4194304 bytes per engine; buffer pool 128 pages; registered I/O buffers off.
…
INFO luradb::server::control: Shard 7: block cache 8192 KiB per engine; MemTable threshold 4194304 bytes per engine; buffer pool 128 pages; registered I/O buffers off.

When the memory that the server takes regardless of load exceeds half of resources.memory_budget, the start logs a warning that lists the parts.

The shard count belongs to the data directory

The data of a domain lies in the files of its shard, so the shard count is part of the layout of the data directory. The first start records it in the system store. With multicore.shards at 0, an existing directory keeps the count it was written with, also after an update or a move to a host with another number of cores; the table above applies to new directories only. When the host would pick another count, the start logs:

WARN luradb::server::layout: Data shards: 1, as the data directory was written; this host would use 2. Writes stay on 1 core(s) until you stop the server and run: luradb --migrate-shards 2

A count that you set explicitly and that differs from the stored one refuses the start, and nothing changes on disk:

Error: shard count changed (stored 2, configured 3) — run `luradb --migrate-shards 3`

luradb --migrate-shards N rewrites the data directory for N shards while the server is stopped, more or fewer, down to 1, and moves key-value, JSON and relational data alike. A domain may end up on another shard, since the shard of a domain follows the count. The procedure, with its preconditions and what happens when it is interrupted, is in Upgrading.

Related settings multicore.shards · multicore.frontend_workers · multicore.reader_threads · multicore.reader_queue_capacity · multicore.shard_queue_capacity · multicore.cpu_offload_threads · buffer_pool.pool_size ·

lsm.memtable_size_threshold