Architecture overview¶

Document role: High-level system architecture narrative. Scope and explicit non-goals: product trade-offs.

liel is a portable external brain for LLMs and local AI tools. Internally, that memory is persisted as a property graph inside a single file (.liel). There is no server process; you carry the library and the .liel file with the workflow that needs the memory.

In other words, "property graph database" describes the storage engine, while "portable external brain" describes the product promise.

Logical model — how users see nodes, edges, labels, and properties.
Physical model — how those map to disk (pages, fixed-size slots, adjacency lists).
Durability — the role of the WAL in making crashes recoverable.

The byte-level layout (offset table) is consolidated in format spec. The decisions that fix the format and the explicit product trade-offs live in product trade-offs.

Architecture at a glance¶

Agent memory layer in the diagram is the role that liel provides: structured, relationship-centric memory. It is not a separate product sitting above liel.

The LLM is optional; the graph is the durable story.

flowchart TB
  agent["Layer 1: Local AI Agent\nplanner, reasoning, tool use"]
  mem["Layer 2: Agent memory layer (concept)\nentities + relations"]
  subgraph L [Layer 3: liel - three peer surfaces, one engine]
    direction LR
    rust["Rust core\ngraph storage, traversal, persistence"]
    py["Python (PyO3)\ninspection, scripting, integration"]
    primary["Primary interface\nliel-demo, MCP tools"]
  end
  store["Layer 4: single-file graph store: .liel"]
  agent -->|memory operations| mem
  mem --> L
  rust --> store
  py --> store
  primary --> store

If your Markdown viewer does not render Mermaid, the same idea in plain text is: agent -> memory (entities + relations) -> liel (Rust + Python + primary surfaces) -> one .liel file.

Logical model (property graph memory)¶

Node — id (auto-assigned u64), one or more labels, optional properties (key/value map).
Edge — id (auto-assigned), source and target nodes, type (label-equivalent), optional properties.
Multiple edges of the same type can connect the same pair of nodes.

For a stakeholder-oriented capability map, see the capability matrix. For the API surface and limits, see the feature list.

Physical model: pages and slots¶

A .liel file is treated as a sequence of fixed 4096-byte pages. The first 128 bytes of page 0 are the file header; the remaining 3968 bytes of page 0 are currently unused. The WAL lives at a fixed location: a 4 MiB reservation starting at byte offset 4096 (page-aligned). Nodes and edges live in fixed-size slots (64 B and 80 B respectively); variable-length data — label strings and properties — lives in separate property extents, referenced from the slot by absolute file offset.

This low-level layout matters because the product's portability depends on a single file that is easy to move, copy, back up, and reopen later as the same memory.

Separating fixed-size slots from variable-length payload makes slot arrays scannable at a constant stride, which keeps the implementation simple. The numeric details and field layouts are owned exclusively by format spec.

Adjacency lists¶

Connectivity is represented as singly linked lists. Each node carries the ID of its first outgoing edge and its first incoming edge; each edge carries the ID of the next edge that shares the same endpoint (next). Traversal does not need an RDB-style JOIN — neighbours are reached at a cost proportional to the degree.

Deleting an edge requires re-linking the list to splice the edge out (see src/graph/edge.rs).

WAL and commit (how durability works)¶

Writes are first appended to the Write-Ahead Log at page granularity. On commit the modified pages are made durable and the WAL is cleared. On startup, any leftover complete WAL is replayed to restore the data pages.

This section is the architectural sketch only. The user-facing durability contract lives in reliability, and the byte layout of WAL entries lives in format spec §6.

Query and execution model¶

liel intentionally has no query planner or Cypher-like language. The public query model is the Python-first API:

direct ID lookup for get_node / get_edge
adjacency-list traversal for out_edges / in_edges / neighbors
BFS-based traversal for bfs, dfs, and directed unweighted shortest_path
QueryBuilder full scans with optional label prefilter and Python where_ predicate

The in-memory LabelIndex is rebuilt on open() and maps labels to node IDs to reduce node scan candidates. It is not a general property index. Arbitrary property predicates remain scans in the current Beta series.

Consistency and concurrency model¶

The consistency model is deliberately narrow:

one writer process owns a .liel file at a time
open() starts an implicit transaction
commit() is the durability boundary
rollback() discards uncommitted dirty pages
transaction() provides a commit-on-success / rollback-on-error context
leftover complete WAL is replayed on the next open()
unsafe double-writer opens fail closed with AlreadyOpenError

This is a single-file embedded store, not a server database. Multiple peer writers, network filesystem semantics, and server-style scheduling are outside the contract. The operational details live in reliability and single-writer guard.

API contract¶

The Beta public contract is Python-first. The compatibility surface is liel.open, the documented GraphDB methods, Node / Edge, QueryBuilder, transaction types, merge reports, and the GraphDBError exception hierarchy.

The Rust modules are the implementation boundary for the Python package. They are kept small and testable, but they are not yet promised as a stable external Rust API in the same way as the Python package surface.

AI memory contract¶

The product promise is not "a complete graph database server." It is a durable, portable graph substrate for AI memory:

store facts, decisions, tasks, sources, files, and tool results as graph records
preserve relationships and provenance across sessions
expose the same .liel file through Python and the optional MCP server
support append/merge style memory writes with explicit commit boundaries

liel does not provide semantic vector retrieval, embedding search, reasoning quality guarantees, or multi-agent concurrent writes by itself. Those belong in the application or agent layer above the storage engine.

Software layers (dependency direction)¶

Bottom up: single file → storage (pages, WAL, cache, serialization) → graph (CRUD, adjacency lists, traversal) → query (QueryBuilder) → GraphDB facade → Python bindings.

The mapping between this layering and the actual Rust modules under src/ is maintainer-facing and lives outside the public docs site (see CONTRIBUTING.md for entry points into the codebase).

Start from the design overview to navigate design, reference, and guide sections.