Skip to content

Internals

Architecture

How KiteDB is structured internally

KiteDB is built in three layers. The query layer is the API you call, the graph layer manages nodes, edges, and transactions, and the storage layer keeps everything in a single file on disk.

The three layers

Query layer

Fluent API, type inference, schema validation

db.insert(user).values({...})

Graph layer

Nodes, edges, traversal, transactions

create_node()add_edge()out_edges()

Storage layer

Memory-mapped file, crash recovery

Snapshot (CSR)DeltaWALKey index

Query layer

This is what you interact with. It provides the Drizzle-style API with full TypeScript type inference. When you write db.insert(user).values(...), the query layer validates your schema, converts TypeScript types to storage types, and calls into the graph layer.

Graph layer

Manages the graph abstraction: nodes with properties, edges between nodes, and traversals. Handles transaction boundaries and coordinates reads between the snapshot and delta.

Storage layer

Stores data in a format built for graph operations. It is organized around the snapshot + delta model, which separates an immutable snapshot on disk from the changes made since it was written.

What happens when you insert a node

Here is the path a single insert takes:

TypeScript
const alice = db  .insert(user)  .values({ key: 'alice', name: 'Alice', age: 30 })  .returning();
1

Query layer

Validates that user has its required properties
Converts age: 30 to the internal I64 type
Calls the graph layer: create_node(...)
2

Graph layer

Begins a transaction (if one isn't already open)
Allocates a new NodeId from a monotonic counter
Records the node in transaction state
On commit, writes to the WAL and the delta
3

Storage layer

WAL: appends a CREATE_NODE record for durability
Delta: adds the node to the created_nodes map
Later: a checkpoint merges it into the snapshot

What happens when you read

Reads merge data from two sources:

TypeScript
const alice = db.get(user, 'alice');
1

Key index lookup

Check delta.key_index for recent changes
If not found, check the snapshot's hash-bucketed index
Returns the NodeId
2

Property fetch

Check delta.modified_nodes for changes
Fall back to the snapshot for unchanged properties
Merge both and return the result

Returns the latest committed data

Why this design

Design choiceBenefit
Snapshot + deltaReads don't block writes. The snapshot is immutable and the delta is small.
CSR format for edgesTraversals read contiguous memory, which makes good use of CPU caches.
WAL for durabilityCommitted data survives crashes. Recovery only replays the WAL written since the last checkpoint.
Single filePortable, atomic operations, simpler deployment.
Memory-mapped I/OThe OS page cache keeps hot pages in memory. Uncompressed sections are read in place.

Next steps