How I gave Claude Code memory without a vector DB

A small hook injects at most 4 notes from my Obsidian vault into every prompt. No embeddings, no database, and it knows when to stay silent.

Jul 18, 2026

TL;DR

Claude forgets everything between sessions, and the popular fixes are vector databases and memory frameworks.
I shipped a bounded recall hook instead: at most 4 relevant notes from my Obsidian vault per prompt, and silence when nothing truly matches.
Under roughly 50,000 notes, plain lexical search with a hard budget beats a vector database. Your notes are already the memory.

Hand-drawn sketchnote titled "Memory without a database". A stack of four notes flows into a terminal window labeled AI agent with an arrow reading "4 notes max", while a crossed-out database cylinder is marked "not needed". Tagline: your notes are the memory.

You can give Claude Code persistent memory with markdown files you already have. No vector database, no memory framework, no new service. Here is the version I run every day, and why the boring approach wins.

I run Claude Code all day. Every session used to start the same way: re-explaining who I am, what I decided last month, which approach I already rejected. The model is brilliant and amnesiac at the same time.

The standard advice for fixing this is heavy. Spin up a vector database. Add an embeddings pipeline. Install a memory framework. Mem0 alone has 48,000 GitHub stars and $24M in funding riding on that advice.

I took the opposite bet. My memory layer is a folder of markdown notes and one small hook. It has been running on my vault of 1,700+ notes, and it is free and open source.

The memory problem nobody names

Sketchnote comparing two prompts. Without a budget, a huge memory blob squeezes the task into a corner. With a recall budget, memory is a thin capped strip labeled "memory: 4 notes" and the task fills the space.

Most “Claude memory” guides are feature tours. They list the layers, show the config flags, and stop.

The unsolved problems sit one level deeper. Context inflation: memory tokens crowd out task tokens until the model reads more about you than about the work. Memory poisoning: one bad write and the agent believes it forever. Stale beliefs: the note says X, reality moved to Y months ago.

Every one of these is a retrieval problem, not a storage problem. Storing knowledge is easy. Deciding what deserves to enter the prompt, and how much, is the hard part.

What Anthropic gives you out of the box

Sketchnote of a three-shelf bookcase: a chat bubble for chat memory, a single MEMORY.md document holding 200 lines, and a wide row of notes labeled "your vault, 1,700 notes", bracketed as the subject of this post.

Claude ships three memory layers now. Chat memory in the apps, which reached free users earlier this year. Projects, which scope context to a workspace. And in Claude Code, Auto Memory: on by default, it writes a MEMORY.md per project and loads the first 200 lines, about 25KB, into the system prompt.

This is genuinely useful, and I keep it on. But it remembers the project, not you. It holds “this repo uses ruff with line-length 100”, not the decision you made in March, the person you met at that call, or the approach you already tried and killed.

That second kind of memory lives in my notes. I have written before about what belongs in a CLAUDE.md and the answer is: not this. Your accumulated knowledge does not fit in 200 lines.

A recall budget, not a database

Sketchnote of a horizontal axis with a tick at 50,000 items. Left of the tick, note cards labeled "text search wins"; right of it, a database cylinder labeled "vectors win". A small dot near zero marks "your vault: 1,700".

Here is the claim that made me build instead of install: below a threshold, vector databases are overhead.

One 2026 analysis puts the crossover near 50,000 stored items. Under that, lexical search (BM25 plus a simple index) beats a vector setup on latency and on operability. Even Anthropic’s own managed agents default to filesystem memory, not a vector store.

A personal vault sits far below 50,000 notes. Mine is around 1,700 after two years of daily use. At that scale the expensive question is not “can I find related notes”, it is “how much found material should I inject”.

So the design flips. The scarce resource is prompt space, not recall. What Claude Code needs is a recall budget.

The hook: 4 notes, 900 characters, or silence

Sketchnote pipeline in four numbered steps: your prompt, a magnifying glass searching the vault, a funnel capping output at max 4 notes, and injection into the AI agent terminal. A side arrow below the funnel reads "no match: silence".

The implementation is one UserPromptSubmit hook. On every prompt, it searches the vault and injects a small brief of the most relevant notes. Four rules make it work:

Bounded. Hard cap of 4 notes, about 900 characters total. Memory can never crowd out the task. That is the whole answer to context inflation.

Abstaining. If confidence is low, it injects nothing. Silence beats noise. A memory system that always says something is a noise generator with a database attached.

Fail-closed. Any error and it exits silently. Recall must never break a prompt.

Observable. Every inject and every abstain decision is logged to a JSONL file, so I can audit what my agent was told and when.

There is a fifth rule hiding in the design: no per-prompt index rebuild. An earlier fork of this idea re-indexed the whole vault on every prompt, which means latency grows with vault size. The hook reuses the index the vault already maintains, so a prompt costs the same at 1,700 notes as it did at 200.

The search underneath is the same one my vault uses everywhere: lexical scoring with freshness and supersession reranking. When a note is superseded by a newer decision, the old one steps back automatically. That is the practical answer to stale beliefs, and it is why my Obsidian vault argues with me instead of parroting my past mistakes.

Memory poisoning gets the same structural answer. Writes into the vault go through validation, and because memory is notes, a bad memory is a file I can open, read, and fix. Try doing that with a corrupted embedding.

Nothing runs until you set two environment variables and register the hook. Memory that reads your private notes should be opt-in, twice.

What changed in practice

Sketchnote split panel. Before: a terminal window with a thought bubble asking "who are you?". After: the same terminal connected by arrows to three notes labeled meeting, person, and decision, with a checkmark.

The visible change is small and constant. I ask about a project, and the context of past decisions is just there. I mention a person, and the agent knows our history. No “as I mentioned before” theater, no re-pasting.

A concrete morning: I typed “draft the follow-up for yesterday’s call”. The hook matched three notes: the meeting note from the call, the person’s card, and a decision record from May that ruled out the approach I was about to suggest again. Four seconds of retrieval saved me from re-proposing something we killed two months ago.

One thing this system will not do is invent structure. The recall is only as good as the notes, and the notes stay good because the vault maintains itself: new information rewrites old pages, contradictions get flagged, and decisions carry dates. Retrieval and note quality are the same problem wearing two hats.

The invisible change matters more. Because the budget is 4 notes, I never wonder whether my prompt is drowning in memory. Because it abstains, unrelated prompts stay clean. Because everything is logged, I have caught bad matches and fixed the notes, not the model. The same pattern powered the LLM wiki experiment that this system grew out of: the notes are the product, the AI is the librarian.

Honest limits: this is personal-scale memory. If you are building agent memory for millions of users, the vector people are right, and you should hire them. For one builder with one vault, they are solving a problem you do not have.

The other limit is discipline-shaped. If you do not write things down, there is nothing to recall. The hook made my notes more valuable, which made me write more of them. That loop was the point all along.

Try it

The hook ships inside obsidian-second-brain, MIT licensed, on GitHub at github.com/eugeniughelbur/obsidian-second-brain. Point it at any folder of markdown notes, set the two variables, register the hook, and your next Claude Code prompt starts with your own knowledge behind it.

Frequently asked questions

How do I give Claude Code persistent memory?
Three options: Claude Code’s built-in Auto Memory (on by default, per-project MEMORY.md), a memory framework with a vector database, or a recall hook over your existing notes. This post covers the third: a UserPromptSubmit hook that injects up to 4 relevant notes per prompt.

Does Claude Code have built-in memory?
Yes. Auto Memory writes a MEMORY.md per project and loads the first 200 lines (about 25KB) into the system prompt. It captures project conventions well, but it is not designed to hold your accumulated decisions, people, and research.

Do I need a vector database for AI agent memory?
Below roughly 50,000 stored items, no. Lexical search with a hard injection budget is faster to run and easier to operate. A personal notes vault is far below that threshold.

What is a recall budget?
A hard cap on how much memory can enter a prompt. Mine is 4 notes, about 900 characters. The cap solves context inflation: memory can inform the task but never crowd it out.

Can Claude read my Obsidian vault?
Yes, locally. The hook searches the vault files on your machine and injects short excerpts into the prompt. Nothing is uploaded anywhere; the vault stays plain markdown on disk.

Is it safe to give an AI hook access to my notes?
The hook is double-gated (two environment variables must be set), fail-closed (any error exits silently), and logged (every inject or abstain is written to a JSONL audit file). You can read every decision it made.

Key takeaways

Agent memory is a retrieval problem, not a storage problem. Your notes already store the knowledge.
Below about 50,000 items, lexical search plus a hard budget beats a vector database on latency and operability.
A recall budget (4 notes, ~900 characters) solves context inflation by construction.
A memory system must be allowed to say nothing. Abstaining beats injecting noise.
Supersession-aware search answers stale memory: replaced decisions step back automatically.
Claude Code’s Auto Memory remembers the project. A vault recall hook remembers you. Run both.

About the author

Eugeniu Ghelbur is an AI Automation Engineer and the creator of obsidian-second-brain, an open-source Claude Code skill starred by over 3,300 developers on GitHub. He builds production AI systems for marketing and sales workflows, and his personal research covers AI agent memory, AI-first knowledge management, and second brains that maintain themselves. He writes The AI Operator (theaioperator.io), a weekly systems newsletter where every post documents a working build, including this one. The fastest way to reach him is e.ghelbur@gmail.com, and his open-source work lives at github.com/eugeniughelbur.

Discussion about this post

Ready for more?