Design Philosophy¶

This page explains the core principles behind memsearch and how they differentiate it from other agent memory solutions.

Markdown is the Source of Truth¶

The foundational principle of memsearch: markdown files are the canonical data store. The vector database is a derived index -- it can be dropped and rebuilt at any time from the markdown files on disk.

graph LR
    MD["Markdown Files<br>(source of truth)"] -->|index| MIL[(Milvus<br>derived index)]
    MIL -->|lost or corrupted?| REBUILD["memsearch index<br>(full rebuild)"]
    REBUILD --> MIL

    style MD fill:#2a3a5c,stroke:#e0976b,color:#a8b2c1
    style MIL fill:#2a3a5c,stroke:#6ba3d6,color:#a8b2c1

Why markdown?

Human-readable. Any developer can open a memory file in any text editor and understand what the agent knows. There is no binary format to decode, no special viewer required.
Git-friendly. Markdown diffs are clean and meaningful. You get full version history, blame, branching, and merge conflict resolution for free -- the same tools you already use for code.
Zero vendor lock-in. Markdown is a plain-text format that has been stable for decades. If you stop using memsearch tomorrow, your knowledge base is still right there on disk, fully intact.
Trivially portable. Copy the files to another machine, another tool, another agent framework. No export step, no migration script, no schema translation.

Why NOT a database as the source of truth?

Opaque. Database files are binary blobs that require specific software to read. If the tool disappears, so does easy access to your data.
Vendor lock-in. Each database engine has its own storage format, query language, and migration tooling. Switching costs are high.
Fragile. Database corruption, version incompatibilities, and backup complexity are real operational concerns for what should be a simple knowledge store.

In memsearch, the vector store is an acceleration layer -- nothing more. If the Milvus database is lost, corrupted, or simply out of date, a single memsearch index command rebuilds the entire index from the markdown files.

Cross-Platform Unified Memory¶

This is memsearch's key differentiator: memories written by one agent are searchable from any other.

graph TB
    subgraph "Capture (per-platform)"
        CC["Claude Code<br/>(Stop hook + Haiku)"]
        OC["OpenClaw<br/>(llm_output + agent)"]
        OO["OpenCode<br/>(SQLite daemon)"]
        CX["Codex CLI<br/>(Stop hook + Codex)"]
    end

    subgraph "Shared Memory"
        MD[".memsearch/memory/*.md"]
        MIL[("Milvus<br/>(shared index)")]
    end

    CC & OC & OO & CX --> MD
    MD --> MIL

    style MD fill:#2a3a5c,stroke:#e0976b,color:#a8b2c1
    style MIL fill:#2a3a5c,stroke:#6ba3d6,color:#a8b2c1

All 4 platform plugins write to the same markdown format and use the same Milvus backend. This means:

You can switch between Claude Code and Codex CLI and keep your memories
Team members using different agents can share a knowledge base
There is no per-agent silo -- one memory, every agent

Most competing solutions are single-platform. memsearch treats multi-platform as a first-class design goal.

Hybrid Search for Quality¶

memsearch uses a three-pronged search strategy to deliver the best possible recall:

Dense vector search -- cosine similarity on embeddings captures semantic meaning ("caching solution" matches "Redis TTL")
BM25 sparse search -- keyword matching catches exact terms that embeddings might miss (error codes, config values, function names)
RRF reranking -- Reciprocal Rank Fusion (k=60) merges the two ranked lists into a single result set

This hybrid approach consistently outperforms pure dense search or pure keyword search alone. The BM25 sparse vector is auto-generated by Milvus -- no application-side sparse encoding is needed.

Progressive Disclosure (L1 → L2 → L3)¶

Agent memory recall must balance two competing needs: context quality (give the agent enough to reason) and context cost (don't blow the context window). memsearch solves this with a three-layer progressive disclosure model:

graph LR
    L1["L1: Search<br/>memsearch search<br/>(chunk snippets)"]
    L2["L2: Expand<br/>memsearch expand<br/>(full section)"]
    L3["L3: Transcript<br/>platform-specific parser<br/>(original conversation)"]

    L1 -->|"need more context?"| L2
    L2 -->|"need exact dialogue?"| L3

    style L1 fill:#2a3a5c,stroke:#6ba3d6,color:#a8b2c1
    style L2 fill:#2a3a5c,stroke:#e0976b,color:#a8b2c1
    style L3 fill:#2a3a5c,stroke:#d66b6b,color:#a8b2c1

Layer	What it returns	Cost
L1: Search	Top-K chunk snippets (summary-level)	Low -- only snippets enter context
L2: Expand	Full markdown section around a chunk, including anchor metadata	Medium -- one file section
L3: Transcript	Original conversation turns verbatim (user messages, assistant responses, tool calls)	High -- raw dialogue

The agent starts with L1 (cheap) and drills deeper only when needed. In the Claude Code plugin, the entire recall process runs in a forked subagent (context: fork), so intermediate results never pollute the main conversation.

Why Milvus¶

memsearch chose Milvus as its vector backend for several reasons:

Requirement	Milvus	SQLite/ChromaDB
Hybrid search	Native dense + BM25 + RRF in a single query	Requires separate FTS5 index + application-side merge
Concurrent access	Built for multi-client	Single-writer lock
Scale path	Lite → Server → Zilliz Cloud (same API)	Limited to single machine
Enterprise ready	Production-proven at scale, managed cloud option	Dev/prototype only
BM25 built-in	Auto-generated sparse vectors via Milvus Function	Manual sparse encoding

The three-tier deployment model is key:

graph TD
    A["memsearch"] --> B{"milvus_uri"}
    B -->|"~/.memsearch/milvus.db<br>(default)"| C["Milvus Lite<br>Local .db file<br>Zero config"]
    B -->|"http://host:19530"| D["Milvus Server<br>Self-hosted<br>Docker / K8s"]
    B -->|"https://...zillizcloud.com"| E["Zilliz Cloud<br>Fully managed<br>Auto-scaling"]

    style C fill:#2a3a5c,stroke:#6ba3d6,color:#a8b2c1
    style D fill:#2a3a5c,stroke:#6ba3d6,color:#a8b2c1
    style E fill:#2a3a5c,stroke:#e0976b,color:#a8b2c1

Start with Milvus Lite (zero config, zero install), scale to Zilliz Cloud (zero ops, auto-scaling) when needed -- same API, same code, just change one URI.

Inspired by OpenClaw¶

memsearch follows OpenClaw's memory architecture precisely:

Concept	OpenClaw	memsearch
Memory layout	`MEMORY.md` + `memory/YYYY-MM-DD.md`	Same
Chunk ID format	`hash(source:startLine:endLine:contentHash:model)`	Same
Dedup strategy	Content-hash primary key	Same
Compact target	Append to daily markdown log	Same
Source of truth	Markdown files (vector DB is derived)	Same
File watch debounce	1500ms	Same default

If you are already using OpenClaw's memory directory layout, memsearch works with it directly -- no migration needed.

Comparison with Competitors¶

Feature	memsearch	claude-mem	opencode-mem	true-mem
Cross-platform	4 platforms	Claude Code only	OpenCode only	Claude Code only
Source of truth	Markdown files	SQLite + ChromaDB	SQLite	SQLite
Search	Hybrid (dense + BM25 + RRF)	Dense only + FTS5	Dense only	Dense only
Embedding	Pluggable (6 providers)	Fixed (MiniLM WASM)	Fixed	Fixed
Progressive disclosure	L1 → L2 → L3	Single layer	Single layer	Single layer
Context isolation	Skill in forked subagent	MCP tools in main context	Tools in main context	Tools in main context
Storage format	`.md` (human-readable, git-friendly)	Binary DB	Binary DB	Binary DB
Vector backend	Milvus (Lite → Server → Cloud)	ChromaDB	Built-in	Built-in
API key required	No (ONNX default)	No (WASM)	Varies	Varies

Key advantages:

Cross-platform portability. memsearch is the only solution that works across Claude Code, OpenClaw, OpenCode, and Codex CLI with shared memory.
Transparent storage. Markdown files are human-readable and git-friendly. You can inspect, edit, and version-control your agent's memories directly.
Search quality. Hybrid search (dense + BM25 + RRF) catches both semantic matches and exact keyword matches that pure-dense solutions miss.
Scale path. Milvus Lite for dev, Milvus Server for teams, Zilliz Cloud for production -- same API throughout.
Context efficiency. Progressive disclosure and forked subagent recall minimize context window usage.