Memory & Learning

This page describes how the agentic harness remembers (three-tier memory), how it retrieves the relevant information (vector RAG recall), and how it turns its errors and successes into reusable rules (the scars → learning → lessons loop).

The three-tier memory

The system distinguishes three memories, of increasing granularity and writing discipline.

Tier	Medium	Role
1 — reflex	memory/*.md files	Loaded into every session. Hard rules, feedback, references. Strict, grep-able, one line per entry in the index.
2 — doctrine	Obsidian Zettelkasten (the brain Vault)	The "why": decisions, doctrine, concepts. Exploratory, linked note to note by internal links.
3 — semantic	vector table ps_ac_embedding (pgvector, HNSW cosine index, 1024 dimensions)	Semantic index of everything above plus the business tables. This is the engine behind RAG recall.

Tier 1 — reflex

A few hundred Markdown files (feedback_*, user_*, project_*, reference_*), indexed by a master file. They are injected into the context of every session: this is the "reflex" memory, loaded without an explicit recall. It is reserved for genuinely high-impact doctrines — each index entry fits on a single line.

Tier 2 — doctrine (Zettelkasten)

The brain Vault gathers several subfolders: permanent (consolidated), inbox (draft drop zone), moc, daily, fleeting. Notes are linked by wikilinks, with a YAML frontmatter type / status.

Promotion rule: inbox/ is the drop zone where the intelligence writes freely; permanent/ is never modified without an explicit order from the Founder. No auto-merge inbox → permanent or inbox → memory: consolidation stays a human gesture.

Tier 3 — semantic (pgvector)

A single vector table carries the semantic index, with an HNSW index in cosine distance and a uniqueness constraint on the (source_type, source_id) pair. Vectors are 1024-dimensional, produced by the canonical embedding provider.

Indexing

Two automates feed tiers 2 and 3. Do not confuse them: one fills a relational table (structured search / hub UI), the other fills the vector table (semantic recall).

• Zettelkasten indexer → sy_brain_note. Walks the Vault subfolders, parses the frontmatter and extracts the wikilinks. Sync is incremental by comparing modification times: UPSERT on the path, deletion of orphan rows whose file no longer exists. Cron every 15 minutes, idempotent.
• Embedding synchroniser → ps_ac_embedding. Embeds the sources then UPSERTs. Deduplication by SHA-256 content hash (skip if the source pair already has the same hash). MVP chunking "one file = one chunk", truncated at 16,000 characters. Embedded in batches of 50 entries.

The indexed sources and their source_type:

source_type	Origin
cicatrice / victoire	the scars table (failures vs successes, by kind)
memory	memory/*.md files (excluding the master index)
brain-<subfolder>	notes from the brain Vault
doctrine / chantier / agent / drill / conduite	active business tables
iteration	ReAct triplets (reason / action / observation)

Immediate reindexing: a scar engraved during a ReAct loop is re-embedded on the fly by a dedicated function, without waiting for the nightly sync — it becomes RAG-searchable right away.

RAG recall

A text query is turned into a vector, then compared by cosine distance to the stored embeddings. The flow:

text query ──embed──> vector
   │
   ▼
SELECT source_type, source_id, content_chunk, metadata,
       1 - (embedding <=> vec) AS similarity
FROM ps_ac_embedding
[WHERE source_type = ANY(scope)]      -- optional scope filter
ORDER BY embedding <=> vec            -- HNSW cosine distance
LIMIT fetch_k
   │
   ▼
re-rank importance × recency ──> final top-K ──> bump the recall counter

• Scopes exposed to the user: iterations, cicatrices, victoires, memory, brain, doctrine, chantiers, agents, drill, conduite, all.
• Importance × recency re-rank: for scars only, the system over-fetches then weights by similarity × (1 + importance/10) × exp(-age_days/365). A fresh, important scar is boosted; an old, minor scar fades away.
• Anti-forgetting bump: every served scar has its recall counter incremented and its last-recall date updated. The archival cron only prunes what has never been recalled.
• Score reading: above 0.75 highly relevant; between 0.65 and 0.75 relevant; below that, validate manually.

The learning loop

This is the heart of capitalisation. An error (or a reproducible success) becomes data, then a concrete suggestion, then — after human validation — a rule loaded into every future session.

ReAct / chantier
      │ error resolved (or reproducible success)
      ▼
the scars table  (kind = failure | victory ; the "check" = the lesson)
      │ immediate reindexing into pgvector
      │ importance scoring ; learnable flag
      ▼
learning engine  (LLM: learnable scar → suggestion)
      │ INSERT suggestion (status = pending)
      ▼
hub: feed / apply / patch   (HUMAN VALIDATION)
      │ status → applied | refused | modified
      ▼
audit log  +  real application in the destination
      ▼
loop closed: the rule sits in memory/ or a CLAUDE.md → next session's reflex

The scars table

The single source of lessons. The kind field separates failures and successes in the same table. The check_added field carries the lesson: for a failure, what was added to avoid a repeat; for a victory, the pattern to reproduce. The importance, scored by an LLM, weights the recall. The boolean learnable is the gateway into the learning engine.

The learning engine

Pipeline: it fetches the learnable scars that do not yet have a suggestion, sorted by importance; it strips personal data before any LLM call (and re-filters the output, defence in depth); the LLM returns a JSON {destination, text, diff before, diff after}; a suggestion is inserted with status pending.

The six possible destinations for a suggestion:

Destination	Meaning
claude_md	rule to add or modify in a CLAUDE.md directives file
settings_json	hooks / permissions / variables config in the harness settings
memory	persistent tier-1 note (reflex)
brain_inbox	new Zettelkasten note (concept / doctrine)
skill	create or improve an invocable skill
chantier_outil	new chantier or automate

No auto-application: the engine only proposes. Validation goes through the hub (feed, apply, patch), and every validated application leaves a trace in an audit log.

Post-chantier lessons

A dedicated agent, Montessori, drafts a lesson when a chantier closes. It loads an audit bundle (meta, travaux, tasks, latest ReAct iterations, linked scars, truncated run logs), passes it to an LLM with a fixed five-section prompt: estimated vs actual token delta, top 3 recurring errors, top 3 patterns to standardise, one candidate doctrine, one tool to create. An ROI safeguard refuses chantiers that are too small. The output lands as a draft in brain/inbox/; promotion to the consolidated tier stays a human gesture.

From patterns to skills

A second pipeline turns the recurring patterns observed in ReAct loops into skill proposals. A weekly detector passes a batch of completed iterations to an LLM that spots patterns present across at least three tasks and worth a skill. Each non-duplicate proposal is inserted with status pending; an hourly monitor alerts the Founder if the queue exceeds a threshold. A proposal validated in the hub is promoted to a native skill, invocable from then on.

The nightly dream

A consolidation automate runs at night: it detects scars repeated three or more times to draft a promotion, and compacts transcripts. Its safeguard is strict — never an auto-merge into the consolidated or reflex tiers, and a dry-run mode by default. The same discipline governs the monthly hygiene of tier-1 memory: dead-link detection, archival of obsolete files, hash-based deduplication, all under an anti-overlap lock and with a timestamped report on every pass.

The AI façade

Every AI call in the system goes through a single façade, exposing two canonical functions: embed (text → vector) and complete (mission → text or JSON).

• File-based routing: in auto mode, the façade reads a routing YAML file that maps each use to a provider and a model. Switching provider means changing one word in that YAML — no redeploy, the next run re-reads the config. Fail-soft: a missing or invalid YAML falls back to the default values.
• Embed: the European sovereign provider is canonical; other providers remain pluggable.
• Complete: the Claude invocation goes through the system's agent-invocation mechanism (never a raw command line); HTTP providers apply up to two retries on transient errors (429 / 5xx), with backoff, a 4xx staying permanent.

Anti-leak (P0 rule): no API key ever appears in the code or the documentation. Provider and database credentials are read from the environment, by variable name, from unversioned secret files.

A single source of truth

Above the three memory tiers reigns one rule: the database is the only source of truth. Markdown serves only runbooks and doctrines; every reference that changes lives in the database. This is the discipline that keeps knowledge from contradicting itself as it grows — and that makes the collective an organism that strengthens instead of repeating its faults.