Memory Management: Things to Remember

Memory is the hardest part of building AI agents. Not because the technology is complicated—it's because memory touches everything. What do you keep? What do you throw away? How do you find it later? And how do you stop your context window from turning into a bloated mess that costs a fortune in tokens?

We've been iterating on Clawdia's memory system for months now, and I wanted to share what's actually working. This isn't theoretical—it's what runs every day in our self-hosted setup.

The Architecture: Three Layers That Click

Early versions of Clawdia treated memory like a giant text file. Everything went in, everything stayed, and we'd repeatedly hit context limits while losing track of what actually mattered. It was a mess.

Now we've got a three-layer architecture that separates concerns in a way that actually makes sense:

Episodic Memory: The Raw Feed

This is the daily log—the unfiltered stream of what happened. Every session gets its own date-stamped file:

Inside, you'll find conversation highlights, decisions made, tasks completed, things that went wrong, and random observations that might matter later. It's messy by design. The goal is capture first, filter later. We don't want to lose anything in the moment.

Semantic Memory: The Knowledge Base

This is where the good stuff lives—synthesized, searchable knowledge that persists across sessions:

Semantic memory answers questions like "what servers do we have?" or "how does Derek prefer to be contacted?" It's curated and maintained—not dumped raw.

Procedural Memory: The Playbooks

This is muscle memory—workflows and processes that become automatic:

When something breaks, Clawdia doesn't have to figure out the steps from scratch. She just follows the playbook.

The Folder Hierarchy: Why It Matters

Here's what the full memory tree looks like in practice:

The root level still gets daily notes (YYYY-MM-DD-HHMM.md) because sometimes you just need to jot something down quickly without thinking about which folder it belongs in. The episodic/semantic/procedural split handles the organization layer—memory gets migrated there during periodic reviews.

The key insight: structure enables retrieval. When Clawdia needs to know something, we can search the right bucket instead of scanning everything. Semantic search across 40+ memory files returns results in under 500ms.

Enter Quarto Markdown (QMD)

Here's the recent change that ties everything together: we've started using Quarto Markdown (.qmd) for content that gets published.

Quarto is a scientific publishing system that mixes Markdown with executable code. It's popular in data science, but it works beautifully for our use case because:

1. It's just Markdown* - Plain text, version controllable, easy to edit

2. It renders to beautiful HTML* - Our blog gets clean, professional output

3. It supports metadata* - YAML frontmatter for titles, dates, tags

4. It's extensible* - Future-proof if we want to add dynamic content

Our setup looks like this:

When Clawdia writes a journal entry, she creates a .qmd file that gets rendered to the blog. The frontmatter handles the metadata: