Memory v2 / Step 2 · Wed 29 Apr 2026

Bootstrap ingest plan

Seed ~/cosmo-memory/topics/ from existing knowledge sources. Three-pass Opus pipeline. Shadow-mode review. Dashboard upgrade first.

Bootstrap is the only ingest pass that runs over all existing knowledge before the dream pass takes over. The dream pass after this only sees new episodes from chat turns. Anything not picked up here either:

• Lives forever in the source file (CLAUDE.md, skill docs) and Cosmo has to re-read it on every relevant turn — slow and tokens-expensive, OR
• Gets caught later by the dream pass when the user mentions it again. If the user never mentions it again, it's effectively invisible.

So: we ARE aiming for completeness, but only over a defined corpus. The trick is defining the corpus precisely, then verifying coverage of THAT corpus. We're not aiming for "all knowledge ever" — that's impossible. We're aiming for "every durable fact present in these specific files."

How do we know what we missed? Three verification mechanisms working together. No single one is sufficient.

1. Coverage pass (Pass 2 of the pipeline). Opus reads source files + generated topics, lists facts NOT extracted, categorises as (a) intentionally excluded, (b) ambiguous, (c) genuine miss. Human reviews the miss list.
2. Round-trip pass (Pass 3). Opus reads each topic, traces it back to source paragraphs. Untraceable topics = invented = revise or delete.
3. Use-driven gap detection (deferred). Over time, log router-misses (zero topics matched) and direct-source-reads. Each is a missing-topic signal. Captured as a future task; lands with step 3 dream pass infrastructure.

Programmatic enumeration. The bootstrap script discovers sources at runtime rather than hardcoding paths. Categories:

Total input estimate: 150-250K tokens. Larger than v1 estimate because session-files set was undercounted before.

These are real sources of memory but require their own ingest strategy. Captured as task files in ~/cosmo-memory/tasks/active/ so we don't forget:

• deferred Bear notes — encrypted SQLite, ~hundreds of long-form notes. Different SCHEMA decisions (granularity, tag mapping). Separate ingest later. (Task: Future ingest: Bear notes into cosmo-memory)
• deferred Strava activities — already accessible via healthyst integration. Separate ingest tied to running history. (Task: Future ingest: Strava activities)
• deferred Gmail — needs email-specific extraction rules. Separate ingest later. (Task: Future ingest: Gmail)
• deferred Photos library — separate ingest with vision-based metadata. (Task: Future ingest: Photos library)
• deferred Alternative note-app evaluation — Bear may not be the long-term home. (Task: Evaluate alternative note app)

And the SCHEMA-level exclusions (always):

• always exclude Secrets / credentials. Skip .env entirely. Reference patterns ("VAR lives in cosmo/.env") OK; values not.
• always exclude Code. No JS/Python/HTML files. Bootstrap reads docs, not source.
• always exclude Transient state derivable from runtime (process lists, etc.).

The existing MEMORY.md at ~/.claude/projects/-Users-sahil-cosmo/memory/ is human-curated. Each entry has explicit Why: and How to apply: lines that took real failures to learn. Re-deriving via Opus risks losing those load-bearing rationales.

Migration approach:

1. Preserve the rule text verbatim. "NEVER copy dist files to server" stays exactly as is.
2. Preserve Why: and How to apply: lines intact.
3. Preserve the date when known.
4. Drop the <type> categorisation (user/feedback/project/reference). The old structure organises by category-of-rule. The new system organises by life domain. The type is metadata that doesn't carry semantic weight in topic-space.
5. Re-route by domain. A feedback_h2os_deployment.md becomes a section inside topics/work-h2os.md, OR its own topic file if substantial enough.

This is migration in the sense that matters: facts kept, structure rebuilt around them. Pass 1 of the pipeline handles MEMORY.md with explicit instructions to preserve verbatim rule text + rationale.

One topic per:

• Distinct life domain — health is one, work-h2os is another, activism is a third
• Distinct durable initiative inside a domain — phase-9-comeback, parkruns-history, migraine-history are three separate health topics
• Distinct entity that gets referenced repeatedly — Kathryn (relationship), CCSA (org), AGM action (campaign)
• Distinct procedural reference — h2os-deployment-process, claude-code-keybindings, port-reservations

NOT one topic per:

• Time slice — don't make "April 2026" a topic. Time = episode date, not topic identity.
• Person mentioned once — doesn't get a topic. Wait for dream pass to elevate.
• Generic concept — "running" alone is too broad. Specific (e.g. user's own training phases) or skip.

Splitting heuristic

If a topic file would mix "current state" + "history" of something, split into two: X-current and X-history. The router pulls current-state files way more often; history files only when the user explicitly asks back in time.

If the source doesn't include a date and Pass 1 can't infer with confidence, leave dateless and add to a TODO list for human review. Don't invent precision.

One Opus run isn't enough. Three sequential passes, each with one job:

The current dashboard at http://127.0.0.1:9876 is a viewer. Bootstrap output is 60-100 topic files. Reviewing in a viewer with no review tooling is brutal. So upgrade dashboard FIRST, then run bootstrap.

New features:

• Per-topic review state — a bootstrap_reviewed: approved | needs-revision | delete field stored in frontmatter (or a side-file). Buttons in the dashboard to set state.
• Comments — a bootstrap_notes: field where you leave a one-liner per file ("missing the X fact", "merge with Y", etc.). Editable from the dashboard.
• Source links — each topic shows which source(s) produced it (from Pass 3 metadata). Clickable.
• Coverage view — a tab showing every source file → which topics reference it. Highlights sources with zero topic references.
• Diff vs source — for a topic, show the source paragraphs that fed into it. Lets you spot misinterpretations.

Time: ~2-3 hours of work.

Bootstrap writes to ~/cosmo-memory/topics-draft-YYYY-MM-DD/, NOT directly to topics/. Review happens in the draft dir. Once approved, files move into topics/.

The flow:

1. Run bootstrap script → writes to topics-draft-2026-04-29/
2. Open dashboard → see all draft topics with their review state (initially all "pending")
3. Review each: approved | needs-revision | delete
4. For "needs-revision": edit the file directly via VSCode or leave a note and come back
5. For "delete": click delete, file goes to topics-draft-DATE/.deleted/ (audit trail, not lost)
6. Once everything is approved or deleted: run cosmo-mem bootstrap merge → moves approved files into topics/, archives the draft dir
7. Commit cosmo-memory repo
8. Restart cosmo-agent so the live system picks up the new topics

This is bounded and worth it. The downstream value (router has things to pick from, dream pass starts running on real topics, "what does Cosmo know about me" actually works) is huge.

Captured as task files in ~/cosmo-memory/tasks/active/ so they don't get forgotten:

• future-ingest-bear-notes-into-cosmo-memory
• future-ingest-strava-activities-into-cosmo-memory
• future-ingest-gmail-into-cosmo-memory
• future-ingest-photos-library-into-cosmo-memory
• evaluate-alternative-note-app-replace-bear-long-term
• build-use-driven-gap-detection-log-router-0-topic-returns-direct-source-reads

1. This doc, signed off. ✅ (if you're reading this, we're here)
2. Deploy doc to Cloudflare Pages (sibling to memory-v2.pages.dev).
3. Dashboard upgrade — review state, comments, source links, coverage view, diff-vs-source. (~2-3h)
4. Bootstrap script at ~/cosmo/scripts/cosmo-mem-bootstrap.js. Reads /continue+/catchup skills to enumerate sources, runs Pass 1, Pass 2, Pass 3 sequentially, writes to topics-draft-YYYY-MM-DD/, generates inbox reports. (~1-2h)
5. Dry-run on MEMORY.md only — sanity check the migration approach for ~$1, ~3 min.
6. Full bootstrap run — all sources, three passes. (~30-45 min, ~$11-15)
7. Review via dashboard — mark every topic. (~3-4h spread over a day)
8. Merge into topics/. Archive draft dir. Commit cosmo-memory.
9. Restart cosmo-agent. Live system picks up new INDEX (auto-loaded).
10. Smoke-test via Telegram — "what's my current training plan?", "where do I live?", "what's H2OS?".

Doc lives at plans/memory-v2-step2-bootstrap.html on the memory-v2 branch. Generated Wed 29 Apr 2026.