Storage

How does Areev store grains?

The storage engine is the durable, encrypted persistence layer for all AI memory grains in the Areev context database.

Grains are written as encrypted .mg blobs, keyed by the grain’s SHA-256 content hash for content-addressed storage. The engine is tuned for high write throughput and durable persistence — a write is acknowledged only once the grain and its indexes are durably committed. The autonomous memory engine reads and writes grains by content hash, which also gives deduplication and integrity verification for free.

Areev keeps several logical data domains: grain blobs, the per-memory wrapped DEKs (data encryption keys), the audit trail, and the entity_latest index (a mapping from entity key to the most recent grain version). AI agent memory operations (write, recall, forget) are atomic and stay consistent across these domains.

The main data domains are:

  • Grains — encrypted .mg grain blobs, addressed by SHA-256 content hash
  • Keys — per-memory data encryption key (DEK) envelopes wrapped by the platform’s managed key service
  • Audit — append-only audit trail events
  • Full-text index — BM25 search index over grain content
  • Vector index — HNSW index of embedding vectors for similarity search
  • Graph index — entity-relation triples for graph traversal

How does full-text search work?

Areev indexes grain content using BM25 ranking and supports phrase queries, boolean operators, and field-scoped searches.

When the autonomous memory engine writes a grain, it extracts the text content and inserts it into the full-text index along with metadata fields (subject, namespace, grain type). The engine tokenizes the text, builds an inverted index, and stores term frequencies and positions for BM25 scoring. The context database queries this index during recall operations that include a text query component.

The full-text index is maintained independently of the grain store and updated in the background as grains are written. AI memory grains stay searchable until they are explicitly forgotten — crypto-erasure removes both the stored grain and its full-text index entry atomically.

# Search via HTTP API
curl -s -X POST https://acme.areev.ai/api/memories/default/recall \
  -H "Content-Type: application/json" \
  -d '{"query": "quarterly revenue", "limit": 10}'
# Python full-text search
import requests
results = requests.post("https://acme.areev.ai/api/memories/default/recall", json={
    "query": "quarterly revenue",
    "limit": 10
}).json()
for grain in results["items"]:
    print(grain["subject"], grain["score"])

How does vector similarity search work?

The HNSW (Hierarchical Navigable Small World) index stores grain embedding vectors and serves approximate nearest-neighbor (ANN) search.

When a grain includes an embedding vector (either provided by the caller or computed by an LLM enrichment hook), the AI agent memory engine inserts the vector into the HNSW index. Approximate nearest-neighbor queries run in sublinear time relative to the dataset size. The context database uses cosine similarity as the distance metric. For small datasets the engine uses an exact cosine scan; as the dataset grows it switches to the HNSW index automatically.

Vector search results include a similarity score that the recall engine fuses with BM25 scores (via RRF) when both a text query and a vector query are present in a hybrid recall request.

# Vector similarity search via HTTP
curl -s -X POST https://acme.areev.ai/api/memories/default/recall \
  -H "Content-Type: application/json" \
  -d '{"vector": [0.1, 0.2, 0.3, ...], "top_k": 5}'

How does the graph index store entity relations?

The graph index stores entity-relation triples (subject, predicate, object) in multiple orderings, enabling fast lookups starting from any element of the triple.

Entity-type AI memory grains automatically generate triples that the context database inserts into the graph index. For example, a grain with subject user-123 and a works_at relation to acme-corp is reachable by any partial pattern: “find all entities that work at Acme Corp” (starting from object), “find all relations for user-123” (starting from subject), or “find all works_at relations” (starting from predicate) — each resolved against a sorted index, not a full scan.

The entity_latest index complements the graph index by maintaining a pointer from each entity key to its most recent grain version. This supports use cases where agents need the current state of an entity rather than its full history. The autonomous memory engine updates entity_latest atomically with each grain write, so it always reflects the most recent version.