McpEngramMemory.Core 0.4.1

MCP Engram Memory

A cognitive memory MCP server that provides an LLM with namespace-isolated vector storage, k-nearest-neighbor search (cosine similarity), a knowledge graph, semantic clustering, lifecycle management with activation energy decay, and physics-based re-ranking. Data is persisted to disk as JSON with debounced writes.

Quickstart

Option 1 — dotnet (recommended)

git clone https://github.com/wyckit/mcp-engram-memory.git
cd mcp-engram-memory
dotnet build

Add to your MCP client config (Claude Code, Copilot, etc.):

{
  "mcpServers": {
    "engram-memory": {
      "command": "dotnet",
      "args": ["run", "--project", "/path/to/mcp-engram-memory/src/McpEngramMemory"]
    }
  }
}

Option 2 — Docker

docker build -t mcp-engram-memory .
docker run -i -v memory-data:/app/data mcp-engram-memory

Option 3 — NuGet (embed in your app)

dotnet add package McpEngramMemory.Core --version 0.3.0

That's it. The server exposes 38 MCP tools. To reduce tool count, set MEMORY_TOOL_PROFILE:

{
  "env": { "MEMORY_TOOL_PROFILE": "minimal" }
}

See examples/ for ready-to-use config files.

Architecture

graph TD
    subgraph MCP["MCP Server (stdio)"]
        Tools["11 Tool Classes<br/>38 MCP Tools"]
    end

    Tools --> CI["CognitiveIndex<br/><i>Thin facade: CRUD, locking, limits</i>"]

    CI --> RE["Retrieval"]
    CI --> GR["Graph"]
    CI --> LC["Lifecycle"]
    CI --> IN["Intelligence"]
    CI --> EX["Experts"]
    CI --> EV["Evaluation"]

    subgraph RE["Retrieval Engine"]
        VS["VectorSearchEngine<br/><i>Two-stage Int8→FP32</i>"]
        HS["HybridSearchEngine<br/><i>BM25 + Vector RRF</i>"]
        HW["HnswIndex<br/><i>O(log N) ANN</i>"]
        QE["QueryExpander"]
        TR["TokenReranker"]
        VQ["VectorQuantizer<br/><i>SIMD Int8</i>"]
    end

    subgraph GR["Knowledge Graph"]
        KG["KnowledgeGraph<br/><i>Directed edges, BFS</i>"]
    end

    subgraph LC["Lifecycle Engine"]
        LE["LifecycleEngine<br/><i>Decay, state transitions</i>"]
        PE["PhysicsEngine<br/><i>Gravitational re-ranking</i>"]
    end

    subgraph IN["Intelligence"]
        CM["ClusterManager"]
        AS["AccretionScanner<br/><i>DBSCAN</i>"]
        DD["DuplicateDetector"]
    end

    subgraph EX["Expert Routing"]
        ED["ExpertDispatcher"]
        DM["DebateSessionManager"]
    end

    subgraph EV["Evaluation"]
        BR["BenchmarkRunner<br/><i>MRR, nDCG, Recall@K</i>"]
        MC["MetricsCollector<br/><i>P50/P95/P99</i>"]
    end

    CI --> NS["NamespaceStore<br/><i>Lazy loading, partitioned</i>"]
    NS --> SP["Storage Provider"]

    subgraph SP["Storage"]
        PM["PersistenceManager<br/><i>JSON + SHA-256 checksums</i>"]
        SQ["SqliteStorageProvider<br/><i>WAL mode</i>"]
    end

    NS --> EMB["OnnxEmbeddingService<br/><i>bge-micro-v2, 384-dim</i>"]

    subgraph BG["Background Services"]
        BG1["EmbeddingWarmup<br/><i>startup</i>"]
        BG2["DecayService<br/><i>every 15 min</i>"]
        BG3["AccretionService<br/><i>every 30 min</i>"]
    end

The Core Memory Loop

INGEST → INDEX → RETRIEVE → REINFORCE → DECAY → SUMMARIZE/COLLAPSE
   │                  │          │          │              │
   └── store_memory   │   memory_feedback  │    collapse_cluster
       (embed+upsert) │   (agent feedback) │    (DBSCAN → summary)
                      └── search_memory    └── decay_cycle
                          (k-NN/hybrid)    (activation energy)

Memories move through lifecycle states based on usage:

STM (short-term) ──promote──→ LTM (long-term) ──decay──→ Archived
                   ←─────────────────────────────────── deep_recall
                              (auto-resurrect if score ≥ 0.7)

Project Structure

The solution is split into two projects:

src/
  McpEngramMemory/              # MCP server (Program.cs + Tool classes)
  McpEngramMemory.Core/         # Core library
    Models/                     # CognitiveEntry, SearchResults, MemoryLimitsConfig, etc.
    Services/
      CognitiveIndex.cs         # Thin facade: CRUD, locking, delegates to engines below
      NamespaceStore.cs         # Namespace-partitioned storage with lazy loading
      PhysicsEngine.cs          # Gravitational force re-ranking
      Retrieval/                # Search pipeline
        VectorMath.cs           #   SIMD-accelerated dot product & norm
        VectorSearchEngine.cs   #   Two-stage Int8 screening + FP32 reranking
        HnswIndex.cs            #   HNSW approximate nearest neighbor index
        HybridSearchEngine.cs   #   BM25 + vector RRF fusion
        BM25Index.cs            #   Keyword search index
        QueryExpander.cs        #   IDF-based query term expansion
        TokenReranker.cs        #   Token-overlap reranker (implements IReranker)
        VectorQuantizer.cs      #   Int8 scalar quantization
        IReranker.cs            #   Pluggable reranker interface
      Graph/
        KnowledgeGraph.cs       #   Directed graph with adjacency lists
      Intelligence/
        ClusterManager.cs       #   Semantic cluster CRUD + centroid computation
        AccretionScanner.cs     #   DBSCAN density scanning + reversible collapse
        DuplicateDetector.cs    #   Pairwise cosine similarity duplicate detection
        AccretionBackgroundService.cs
      Lifecycle/
        LifecycleEngine.cs      #   Decay, state transitions, deep recall
        DecayBackgroundService.cs
      Experts/
        ExpertDispatcher.cs     #   Semantic routing to expert namespaces
        DebateSessionManager.cs #   Debate session state + alias mapping
      Evaluation/
        BenchmarkRunner.cs      #   IR quality benchmarks
        MetricsCollector.cs     #   Operational metrics + percentiles
      Storage/
        IStorageProvider.cs     #   Storage abstraction interface
        PersistenceManager.cs   #   JSON file backend with debounced writes
        SqliteStorageProvider.cs #   SQLite backend with WAL mode
tests/
  McpEngramMemory.Tests/        # xUnit tests (458 tests)
benchmarks/
  baseline-v1.json              # IR quality baseline (MRR 1.0, nDCG@5 0.938, Recall@5 0.867)
  baseline-paraphrase-v1.json
  baseline-multihop-v1.json
  baseline-scale-v1.json
  ideas/                        # Benchmark proposals and analysis

NuGet Package

The core engine is available as a NuGet package for use in your own .NET applications.

dotnet add package McpEngramMemory.Core --version 0.3.0

Library Usage

using McpEngramMemory.Core.Models;
using McpEngramMemory.Core.Services;
using McpEngramMemory.Core.Services.Graph;
using McpEngramMemory.Core.Services.Intelligence;
using McpEngramMemory.Core.Services.Lifecycle;
using McpEngramMemory.Core.Services.Storage;

// Create services
var persistence = new PersistenceManager();
var embedding = new OnnxEmbeddingService();
var index = new CognitiveIndex(persistence);
var graph = new KnowledgeGraph(persistence, index);
var clusters = new ClusterManager(index, persistence);
var lifecycle = new LifecycleEngine(index, persistence);

// Store a memory
var vector = embedding.Embed("The capital of France is Paris");
var entry = new CognitiveEntry("fact-1", vector, "default", "The capital of France is Paris", "facts");
index.Upsert(entry);

// Search by text
var queryVector = embedding.Embed("French capital");
var results = index.Search(queryVector, "default", k: 5);

Tech Stack

• .NET 8, C#
• ModelContextProtocol 1.0.0
• FastBertTokenizer 0.4.67 (WordPiece tokenization)
• Microsoft.ML.OnnxRuntime 1.17.0 (ONNX model inference)
• Microsoft.Data.Sqlite 8.0.11 (SQLite storage backend)
• bge-micro-v2 ONNX model (384-dimensional vectors, MIT license, downloaded at build time)
• Microsoft.Extensions.Hosting 8.0.1
• xUnit (tests)

MCP Tools (38 total)

Core Memory (3 tools)

Knowledge Graph (4 tools)

Supported relation types: parent_child, cross_reference, similar_to, contradicts, elaborates, depends_on, custom.

Semantic Clustering (5 tools)

Lifecycle Management (5 tools)

Activation energy formula: (accessCount x reinforcementWeight) - (hoursSinceLastAccess x decayRate)

Admin (2 tools)

Accretion (4 tools)

collapse_cluster reliability behavior:

• If collapse steps complete successfully, the pending collapse is removed and a reversal record is persisted to disk.
• If summary storage or any member archival step fails, the tool returns an error and preserves the pending collapse so the same collapseId can be retried.
• Collapse records survive server restarts and can be reversed with uncollapse_cluster.

Intelligence & Safety (5 tools)

Panel of Experts / Debate (3 tools)

Debate workflow: consult_expert_panel (gather perspectives) → map_debate_graph (define relationships) → resolve_debate (store consensus). Sessions use integer aliases (1, 2, 3...) so the LLM never handles UUIDs. Sessions auto-expire after 1 hour.

Benchmarking & Observability (3 tools)

Five benchmark datasets: four covering generic CS topics (programming languages, data structures, ML, databases, networking, systems, security, DevOps) and one real-world dataset modeled after actual cognitive memory entries (architecture decisions, bug fixes, code patterns, user preferences, lessons learned). Relevance grades use a 0–3 scale (3 = highly relevant).

Maintenance (2 tools)

Expert Routing (2 tools)

Expert routing workflow: dispatch_task (route query) → if miss: create_expert (define specialist) → dispatch_task (retry). The system maintains a hidden _system_experts meta-index that maps queries to specialized namespaces via cosine similarity (default threshold: 0.75). Experts within a 5% score margin of the top match are returned as candidates.

Architecture

Services

CognitiveIndex is a thin facade managing CRUD, locking, and memory limits. Search, hybrid search, and duplicate detection are delegated to stateless engines that operate on data snapshots.

Background Services

Models

Searchable Compression

Vectors use a lifecycle-driven compression pipeline:

• STM entries: Full FP32 precision for maximum search accuracy
• LTM/archived entries: Auto-quantized to Int8 (asymmetric min/max → [-128, 127]) on state transition
• HNSW index: Namespaces with 200+ entries auto-build an HNSW graph for O(log N) approximate nearest neighbor candidate generation
• Two-stage search: Namespaces with 30–199 entries use Int8 screening (top k×5 candidates) followed by FP32 exact cosine reranking
• SIMD acceleration: Int8DotProduct uses System.Numerics.Vector<T> for portable hardware-accelerated dot products (sbyte→short→int widening pipeline)
• Base64 persistence: Vectors are serialized as Base64 strings instead of JSON number arrays, reducing disk usage by ~60%. Legacy JSON arrays are still readable for backwards compatibility.

Persistence

Two storage backends are available, selectable via environment variable:

JSON file backend (default):

• Data stored in a data/ directory as JSON files
• {namespace}.json — entries with Base64-encoded vectors (per namespace)
• _edges.json — graph edges (global)
• _clusters.json — semantic clusters (global)
• _collapse_history.json — reversible collapse records (global)
• _decay_configs.json — per-namespace decay configurations (global)
• Writes are debounced (500ms default) with SHA-256 checksums for crash recovery

SQLite backend (MEMORY_STORAGE=sqlite):

• Single memory.db file with WAL mode for concurrent read/write
• Tables: entries, edges, clusters, collapse_history, decay_configs, schema_version
• Automatic schema migrations (v1→v2 adds lifecycle_state column with backfill)
• Suitable for higher-throughput or multi-process scenarios

Environment Variables

Usage

MCP Server

Configure the MCP server in your client (e.g. Claude Desktop, VS Code):

{
  "mcpServers": {
    "engram-memory": {
      "command": "dotnet",
      "args": ["run", "--project", "/path/to/mcp-engram-memory/src/McpEngramMemory"],
      "env": {
        "MEMORY_STORAGE": "sqlite",
        "MEMORY_MAX_NAMESPACE_SIZE": "10000"
      }
    }
  }
}

The env block is optional. Omit it to use the JSON file backend with no memory limits.

AI Assistant Setup

Each setup below is a single prompt you paste into the respective tool. The AI will read this README, create all config files, and write the custom instructions for you.

Claude Code Setup

Open Claude Code in your project directory and paste:

Set up mcp-engram-memory as my persistent memory system. Do the following:

1. Add the MCP server to my Claude Code config. The server runs via:
   command: dotnet
   args: run --project /path/to/mcp-engram-memory/src/McpEngramMemory

2. Create or update my CLAUDE.md (global at ~/.claude/CLAUDE.md) with these sections:

   ## Recall: Search Before You Work
   - At conversation start, search vector memory using up to 3 parallel agents:
     Agent 1: search_memory in the project namespace for the current task
       (use hybrid: true and expandGraph: true for richer recall)
     Agent 2: search_memory in "work" and "synthesis" namespaces for cross-project patterns
     Agent 3: search_memory with alternative phrasings/keywords in the project namespace
       (use hybrid: true for keyword+vector fusion)
   - For graph-connected knowledge, use expandGraph: true to pull in linked memories
   - Tool selection: search_memory for project context, dispatch_task for cross-domain
     questions, consult_expert_panel for multi-perspective analysis, deep_recall for
     archived knowledge, detect_duplicates and find_contradictions for quality checks

   ## Store: Save What You Learn
   - Store memories after completing tasks, fixing bugs, learning patterns, or receiving
     corrections. Use the project directory name as namespace, kebab-case IDs, include
     domain keywords in text for searchability, and categorize as one of: decision, pattern,
     bug-fix, architecture, preference, lesson, reference
   - Pre-store quality checks: verify text is self-contained, includes domain keywords,
     doesn't duplicate existing memories, and has the correct category
   - When store_memory warns about duplicates: skip if existing is accurate, upsert same
     ID if outdated, or store and link if both are distinct

   ## Expert Routing
   - Use dispatch_task for open-ended questions. If it returns needs_expert,
     call create_expert with a detailed persona, then seed that expert's namespace
   - Lifecycle: promote STM to LTM when recalled 2+ times, documents a stable pattern,
     captures a recurring bug fix, or records a user correction
   - Link related memories with link_memories using: parent_child, cross_reference,
     similar_to, contradicts, elaborates, depends_on

   ## Session Retrospective
   - At the end of significant sessions, self-evaluate: what went well, what went wrong,
     what you'd do differently, key decisions made
   - Store retrospective with: id "retro-YYYY-MM-DD-topic", category "lesson",
     specific actionable lessons (not vague observations)
   - Link retrospectives to related bug fixes, patterns, or decisions
   - Search past retrospectives before starting similar work

Confirm each file you create and show me the final contents.

GitHub Copilot Setup

Open VS Code with Copilot and paste in chat:

Set up mcp-engram-memory as my persistent memory system. Do the following:

1. Create .vscode/mcp.json with a stdio server entry:
   name: engram-memory
   command: dotnet
   args: ["run", "--project", "/path/to/mcp-engram-memory/src/McpEngramMemory"]

2. Create .github/copilot-instructions.md with vector memory instructions:

   ## Recall
   - Before starting any task, use search_memory with the project namespace to recall
     relevant context. Use hybrid: true for keyword+vector fusion and expandGraph: true
     to pull in linked memories. Search for past decisions and bugs before answering.
   - Tool selection: search_memory for project context, dispatch_task for cross-domain
     questions (auto-routes to best expert namespace), consult_expert_panel for multiple
     perspectives, deep_recall for archived/forgotten knowledge, detect_duplicates and
     find_contradictions for memory quality.

   ## Store
   - Store memories after completing tasks, fixing bugs, learning patterns, or receiving
     corrections. Use project directory name as namespace, kebab-case IDs, write text
     with domain keywords for future searchability, categorize as: decision, pattern,
     bug-fix, architecture, preference, lesson, reference.
   - Pre-store quality checks: verify text is self-contained, includes domain keywords,
     doesn't duplicate existing memories, and has the correct category.
   - On duplicate warnings: skip if existing is accurate, upsert if outdated, store and
     link if both are distinct.

   ## Expert Routing
   - dispatch_task routes to experts automatically. If needs_expert is returned, use
     create_expert with a detailed persona description, then populate the expert namespace.
   - Lifecycle: promote STM to LTM when stable and reused across sessions.
   - Link related memories using: parent_child, cross_reference, similar_to, contradicts,
     elaborates, depends_on.

   ## Session Retrospective
   - At the end of significant sessions, store a self-evaluation: what went well, what
     went wrong, lessons learned. Use id "retro-YYYY-MM-DD-topic", category "lesson".
   - Search past retrospectives before starting similar work to avoid repeating mistakes.

Confirm each file you create and show me the final contents.

Google Gemini CLI Setup

Open Gemini CLI and paste in chat:

Set up mcp-engram-memory as my persistent memory system. Do the following:

1. Add the MCP server to my Gemini CLI config (edit ~/.gemini/settings.json):
   name: engram-memory
   command: dotnet
   args: ["run", "--project", "/path/to/mcp-engram-memory/src/McpEngramMemory"]

2. Create GEMINI.md in my workspace root with vector memory instructions:

   ## Recall
   - Before starting any task, use search_memory with the project namespace to recall
     relevant context. Use hybrid: true for keyword+vector fusion and expandGraph: true
     to pull in linked memories. Search for past decisions and bugs before answering.
   - Tool selection: search_memory for project context, dispatch_task for cross-domain
     questions (auto-routes to best expert namespace), consult_expert_panel for multiple
     perspectives, deep_recall for archived/forgotten knowledge, detect_duplicates and
     find_contradictions for memory quality.

   ## Store
   - Store memories after completing tasks, fixing bugs, learning patterns, or receiving
     corrections. Use project directory name as namespace, kebab-case IDs, write text
     with domain keywords for future searchability, categorize as: decision, pattern,
     bug-fix, architecture, preference, lesson, reference.
   - Pre-store quality checks: verify text is self-contained, includes domain keywords,
     doesn't duplicate existing memories, and has the correct category.
   - On duplicate warnings: skip if existing is accurate, upsert if outdated, store and
     link if both are distinct.

   ## Expert Routing
   - dispatch_task routes to experts automatically. If needs_expert is returned, use
     create_expert with a detailed persona description, then populate the expert namespace.
   - Lifecycle: promote STM to LTM when stable and reused across sessions.
   - Link related memories using: parent_child, cross_reference, similar_to, contradicts,
     elaborates, depends_on.

   ## Session Retrospective
   - At the end of significant sessions, store a self-evaluation: what went well, what
     went wrong, lessons learned. Use id "retro-YYYY-MM-DD-topic", category "lesson".
   - Search past retrospectives before starting similar work to avoid repeating mistakes.

Confirm each file you create and show me the final contents.

OpenAI Codex Setup

Open Codex CLI in your project directory and paste:

Set up mcp-engram-memory as my persistent memory system. Do the following:

1. Add the MCP server to my Codex config. Either run:
   codex mcp add engram-memory -- dotnet run --project /path/to/mcp-engram-memory/src/McpEngramMemory
   Or add this to ~/.codex/config.toml (or .codex/config.toml for project-scoped):
   [mcp_servers.engram-memory]
   command = "dotnet"
   args = ["run", "--project", "/path/to/mcp-engram-memory/src/McpEngramMemory"]

2. Create AGENTS.md in the project root with vector memory instructions:

   ## Recall
   - Before starting any task, use search_memory with the project namespace to recall
     relevant context. Use hybrid: true for keyword+vector fusion and expandGraph: true
     to pull in linked memories. Search for past decisions and bugs before answering.
   - Tool selection: search_memory for project context, dispatch_task for cross-domain
     questions (auto-routes to best expert namespace), consult_expert_panel for multiple
     perspectives, deep_recall for archived/forgotten knowledge, detect_duplicates and
     find_contradictions for memory quality.

   ## Store
   - Store memories after completing tasks, fixing bugs, learning patterns, or receiving
     corrections. Use project directory name as namespace, kebab-case IDs, write text
     with domain keywords for future searchability, categorize as: decision, pattern,
     bug-fix, architecture, preference, lesson, reference.
   - Pre-store quality checks: verify text is self-contained, includes domain keywords,
     doesn't duplicate existing memories, and has the correct category.
   - On duplicate warnings: skip if existing is accurate, upsert if outdated, store and
     link if both are distinct.

   ## Expert Routing
   - dispatch_task routes to experts automatically. If needs_expert is returned, use
     create_expert with a detailed persona description, then populate the expert namespace.
   - Lifecycle: promote STM to LTM when stable and reused across sessions.
   - Link related memories using: parent_child, cross_reference, similar_to, contradicts,
     elaborates, depends_on.

   ## Session Retrospective
   - At the end of significant sessions, store a self-evaluation: what went well, what
     went wrong, lessons learned. Use id "retro-YYYY-MM-DD-topic", category "lesson".
   - Search past retrospectives before starting similar work to avoid repeating mistakes.

Confirm each file you create and show me the final contents.

Build & Test

cd mcp-engram-memory
dotnet build
dotnet test

Tests

28 test files with 458 test cases covering: