Chapters

DOC-05 / Technical reference · Chapter 08

Memory & Learning — Three-Level Architecture

This chapter describes the three memory levels of the Synedre OS harness (reflex, Zettelkasten, vector pgvector), the associated indexing automata, and the loop for capitalizing errors into reusable rules.

Memory & learning of the agentic harness

This page describes how the agentic harness remembers (three-level memory), retrieves relevant information (semantic recall via vector similarity) and capitalises on its errors and successes as reusable rules (scars → learning → lessons loop).

1. Three-level memory

The system distinguishes three memory stores of increasing granularity and write discipline.

┌─────────────────────────────────────────────────────────────────┐
│ LEVEL 1 — Reflex memory                                         │
│   Loaded in EVERY session. Hard rules, feedback,                │
│   references. Strict, one entry per line in the index.          │
├─────────────────────────────────────────────────────────────────┤
│ LEVEL 2 — Knowledge base (Zettelkasten)                         │
│   The "why": decisions, doctrine, concepts. Exploratory.        │
│   Draft zone = free deposit; consolidated zone = mandatory      │
│   human validation.                                             │
├─────────────────────────────────────────────────────────────────┤
│ LEVEL 3 — Vector semantic index                                 │
│   HNSW vector index (cosine similarity, 1024 dimensions)        │
│   covering both preceding levels + DB tables.                   │
│   Enables RAG recall (Retrieval-Augmented Generation).          │
└─────────────────────────────────────────────────────────────────┘

Level 1 — Reflex memory

  • An index file lists all active entries (one line per entry, ≤ 200 characters each).
  • Details live in thematic files: experience feedback, user preferences, project references.
  • These files are automatically injected into the context of every session: no explicit recall is required. Only genuine high-impact doctrines appear here — write discipline is strict.

Level 2 — Zettelkasten knowledge base

  • Organised into distinct sub-spaces: permanent notes (consolidated), inbox (drafts), tables of contents, daily logs, ephemeral notes.
  • Notes cross-reference each other via wiki links; a YAML frontmatter indicates the type and status of each note.
  • Promotion rule: the inbox is the only zone where the agent writes freely. The permanent zone is never modified without an explicit order from a human operator. No automatic merge between the two zones is permitted — this is an architectural safeguard.

Level 3 — Vector semantic index

  • Vectors are 1024-dimensional, produced by a dedicated embedding model.
  • The index uses the HNSW (Hierarchical Navigable Small World) structure with cosine similarity, guaranteeing very fast approximate searches even on large corpora.
  • A uniqueness constraint per (source type, source identifier) pair prevents duplicates during updates.

2. Indexing

Two distinct automations feed levels 2 and 3. It is important not to confuse them: the first feeds a structural index of the knowledge base (used by interfaces and criteria-based search); the second feeds the semantic vector index (used for RAG recall).

2.1 Knowledge-base structural indexing automation

This automation traverses all sub-spaces of the knowledge base and maintains a structured representation of each note in the relational database.

  • Scope: permanent notes, inbox, tables of contents, daily logs, ephemeral notes.
  • Processing: extraction of the YAML frontmatter and inter-note links from each file.
  • Incremental synchronisation: the file modification date is compared against the value stored in the database; only modified notes are reprocessed. Deleted notes are removed from the index.
  • Data retained per note: relative path, type, title, tags, full frontmatter, body, outgoing links, size, timestamps.
  • Options: force mode (full re-indexing) and stats mode (count without writing).
  • Cadence: every 15 minutes. The operation is idempotent.

2.2 Embedding and vector synchronisation automation

This automation produces semantic vectors for all sources and inserts them into the vector index. It uses an external embedding model (1024 dimensions).

The indexed sources and their indicative volumes are:

Source type Origin Indicative volume
Scars Errors recorded by the reflex engine (all entries with a description) ~740
Victories Successes recorded by the reflex engine ~4
Reflex memory Level-1 memory files (excluding the main index) ~226
Knowledge base Notes from the various Zettelkasten sub-spaces ~14
Doctrines Active doctrine rules ~25
Projects Ongoing projects ~10
Agents Agent descriptors ~30
Drills Training scenarios ~31
Procedures Active operational procedures ~17
ReAct iterations Reasoning / action / observation triplets from agentic loops ~16

Key mechanisms of the automation:

  • Fingerprint-based deduplication: each chunk receives a SHA-256 fingerprint of its content. If the (type, identifier) pair already exists with the same fingerprint, the chunk is skipped — no call to the embedding service is made.
  • Chunking: in the current version, each document constitutes a single chunk, truncated beyond a character threshold. Finer-grained splitting is planned for a later phase.
  • Batch processing: calls to the embedding service group 50 inputs per request to limit network latency.
  • Insertion: each vector is atomically inserted or updated according to the (type, identifier) uniqueness constraint.
  • Options: rebuild mode (full purge followed by complete re-embedding), source-type filter, dry-run mode (simulation without writing).

2.3 Immediate re-indexing of a scar

When a scar is recorded during an active agentic loop (ReAct cycle), a unit re-indexing procedure is triggered immediately: the scar becomes searchable in the RAG without waiting for the next nightly synchronisation. This mechanism ensures that the agent can benefit from a lesson learned a few seconds after recording it, including within the same run.

2.4 Maintenance cadences

The various memory automations run according to the following schedules:

  • Knowledge-base structural indexing: every 15 minutes.
  • Full vector synchronisation (incremental): daily, early morning. Unit re-indexing of scars (§ 2.3) remains active in parallel for immediacy.
  • Nightly consolidation loop: at night, shortly after midnight — proposes syntheses from the inbox to the permanent zone (human validation required).
  • Pattern detector: weekly (Sunday).
  • Skill-proposal detector: weekly (Sunday) and hourly monitoring.
  • Memory metrics: daily, morning.
  • Monthly consolidation: 1st of the month.
  • Maintenance dashboard: daily, early morning.

Architectural note: none of these automations modifies the permanent zone of the knowledge base without explicit human validation. The entire pipeline is designed to be idempotent: an additional execution produces no undesirable side effects.

Recall RAG

The semantic recall engine allows querying the system's indexed memory via a natural-language query. It operates across three complementary search modes, merged by default into a hybrid mode.

Processing pipeline

The incoming query is processed according to the selected mode:

  • Semantic mode: the query is vectorized by the embedding model (1024-dimensional vectors), then compared against the index using cosine distance. Results are sorted by descending similarity.
  • Lexical mode: words of three characters or more are extracted and combined into an OR query. Matching is computed via text-rank scoring against content indexed in French. This mode calls no external API — no key is required. It reliably catches slugs and proper nouns that the embedding tends to blur.
  • Hybrid mode (default): the two preceding rankings are merged via Reciprocal Rank Fusion (RRF, parameter k=60). The combined score is score = Σ 1/(60 + rank_i). A document absent from one of the two modes receives a null score for that mode, without penalizing the other.

A scope filter common to all three modes allows restricting the search to specific source categories: iteration history, scars, victories, reference memory, doctrine, ongoing work items, agent profiles, protocols, or all sources.

Re-ranking by importance and recency

After the initial selection, an over-fetch (by default three times the number of requested results) enables enriched re-ranking, applied to scars only. The formula is:

final_score = similarity × (1 + importance/10) × exp(−age_days / 365)

Concrete effect: a recent high-importance scar is boosted by a factor of ~2×, while an old low-importance scar sees its score reduced to ~0.4×. This mechanism ensures that fresh, critical learnings surface to the top.

Anti-forgetting: recall counter increment

Each scar actually served has its recall counter incremented by 1 and its last-recalled timestamp updated. This operation is non-blocking: any error it raises is silent and does not disrupt the response. The archiving cron system only purges scars whose recall counter has remained at zero — those recalled at least once are preserved.

Score interpretation

  • In semantic mode: score > 0.75 → highly relevant; 0.65–0.75 → relevant; < 0.65 → requires manual review.
  • In hybrid mode (default): RRF scores operate on a different scale (typically < 0.1). Rely on relative rank, not absolute thresholds.

Recall command options

  • --k: number of results to return.
  • --mode lex|sem|hybrid: search mode (default: hybrid).
  • --scope: source scope to query.
  • --json: machine-readable JSON output.

Automatic trigger before creation

An automatic trigger fires before any new work item is created. It extracts the task title and runs a hybrid search for the four closest results. If the embedding API is unavailable (rate limit reached), lexical mode takes over. Results are injected into the diagnostic stream before creation, in a non-blocking manner. The objective is to avoid reinventing a scar, a doctrine, or a work item already on record — semantic search catches entries that the linear memory index was missing.

Recall evaluation harness

The recall non-regression system is built on an automated evaluation harness. It loads a golden dataset — a set of queries associated with the expected source identifiers, actually present in the vector index — and measures search quality without ever mutating recall counters (strict read-only mode).

Computed metrics

Metric Definition
recall@k Fraction of expected sources found in the top-k, averaged across all queries.
mrr@10 Reciprocal rank of the first relevant result in the top-10 (0 if absent), averaged across queries.
top1_accuracy Fraction of queries for which the first result matches an expected source.

The match between a returned result and an expected source is evaluated by substring inclusion (case-insensitive), making it robust to path prefix variations and identifier format differences.

API rate management

For modes that call the embedding API, a throttle delay of 1.3 seconds is introduced between each call. In the event of a rate-limit response (HTTP 429), exponential backoff is applied for up to five attempts. Lexical mode alone calls no API and requires no delay.

Commands and exit codes

Mode / option Effect Exit code
(no option) Gate mode: compares current metrics against the frozen baseline. Fails if regression exceeds tolerance. 0 (success) / 1 (regression)
--baseline Freezes a new baseline by running evaluation in hybrid mode. 0
--mode-sweep Compares all three modes (lexical, semantic, hybrid) for informational purposes, without gating. 0
--tol <value> Acceptable regression tolerance (default: 0.05).
--json Machine-readable output.
(baseline absent) Warning: no reference baseline found. 2

Operational note: the evaluation harness never modifies database state. It is designed to run in continuous integration with no side effects on anti-forgetting counters or on the metadata of indexed scars.

Learning Loop: Errors, Verifications, Suggestions and Lessons

This is the core of knowledge capitalization. An error (or a success) becomes a data point, then a concrete suggestion, then — after human validation — a rule loaded into all future sessions.

The general flow unfolds in four stages:

  1. A completed task (or a reproducible pattern) is recorded in the lessons registry.
  2. A suggestion engine reads entries marked as learned and produces a structured proposal.
  3. A human operator validates, rejects, or modifies each proposal from the dashboard.
  4. The validated rule is applied to the appropriate destination and becomes active from the next session onward.

4.1 The Lessons Registry

The lessons registry is the single source of truth for all knowledge capitalization. Each entry documents either a resolved failure or a reproducible success. The key fields are:

  • Entry type (kind): distinguishes failures, successes, and a few legacy categories (convention, pattern, bug). The vast majority of entries are resolved failures.
  • The lesson (check_added): for a failure, what was added to prevent recurrence; for a success, the pattern to reproduce.
  • Importance score: automatically computed by a language model at recording time, via the AI routing engine. Non-blocking — if the model is unavailable, the score is populated during the next nightly maintenance run. Importance weights recall frequency (a bonus is granted to the most critical entries).
  • Learnability flag (learnable): gateway for the suggestion engine. Only entries marked true are processed.

Best practice — recording a success: use the dedicated skill (/victoire) rather than direct SQL. The tags field must be supplied as a PostgreSQL literal array ("{a,b,c}"), not a Python list — a lesson learned in production.

4.2 The Suggestion Engine: From Lesson to Proposal

The suggestion engine reads the lessons registry, filters entries marked as learnable and not yet processed, then calls a language model to produce a structured proposal. The pipeline in detail:

  1. Selection: learnable entries absent from the proposal log, sorted by descending importance then by date, capped at 50 per run.
  2. PII scrubbing before sending to the model. The model output is also filtered before insertion — defense in depth.
  3. LLM call: the model returns a JSON object containing the target destination, the suggestion text, and before/after diffs (diff_before / diff_after).
  4. Insertion into the proposal log with status pending. The operation is idempotent; a short delay between calls prevents API rate saturation.

The six possible destinations for a proposal:

Destination Meaning
Orchestrator directive Rule to add or modify in the agent's main directive file
Hooks configuration Permission parameter, hooks, or environment variables
Persistent note (level 1) Immediate reflex injected into each briefing via current memory
Conceptual inbox New Zettelkasten note (concept or doctrine under maturation)
Skill Creation or improvement of an invocable skill
Project or tool New automated project or utility script

4.3 Human Validation and Audit Log

No proposal is applied automatically. The dashboard exposes three operations:

  • Browse the feed (read): list of pending proposals.
  • Apply (write): the proposal moves to status applied and an entry is created in the audit log with the operator identifier and a snapshot of the diff.
  • Reject or modify (write): the proposal moves to status refused or modified.

The audit log retains for each action: the reference to the proposal, the decision author, the timestamp, and the snapshot of the applied diff.

4.4 Post-Project Lessons (Structured Debrief)

In addition to the scar → suggestion loop, a post-project debrief tool allows lessons to be extracted after a project is closed. It is invocable by a dedicated agent via the corresponding skill.

The mechanism loads a full audit bundle: project metadata, work items, tasks, recent iterations, linked scars, and truncated execution logs. Input volume is capped to stay within model limits.

  • ROI guardrail: a project with fewer than three work items is rejected by default (overridable via an explicit flag).
  • Default model: the LLM call goes through the Claude binary via a read-only wrapper (tools Read, Grep, Glob only), with two attempts and a pause between them.
  • Fallback: if the binary is absent or an alternative mode is explicitly requested, a second provider is used, with up to three attempts and exponential backoff.
  • Prompt fixed in five sections: estimated vs. actual token delta, top 3 recurring errors, top 3 patterns to standardize, one candidate doctrine, one tool to create.
  • Output: a lesson note in Zettelkasten format, with status draft, deposited in the Vault inbox.
  • Promotion: never automatic. The operator manually moves the note to permanent memory (RAG) or current memory (reflex) after review.

4.5 Pattern Detection and Skill Proposals

A second pipeline, distinct from the scar loop, transforms recurring patterns observed in work iterations into proposals for invocable skills. This is the capability self-improvement mechanism of the ecosystem.

Task iteration log (completed actions/observations)
      │ weekly scan (Sunday 04:00 UTC)
      ▼
Pattern detector (LLM-first via subprocess, 3 min timeout)
      │ identifies patterns present in ≥ 3 tasks
      │ checks for duplicates (existing skill or already-pending proposal)
      │ INSERT proposal (status='pending', source iterations, LLM evidence)
      ▼
Skill proposal dashboard ── HUMAN validation (founder)
      │ list / validate / reject
      ▼
Promotion to the native skills registry (invocable via /<slug>)
  • Operational detector (weekly cron, Sunday 04:00 UTC): loads recent iterations with status done or pass, passes the batch to the language model which identifies patterns warranting a skill, then inserts proposals into the log after deduplication. Flags --dry-run and --limit available.
  • Proposal monitor (hourly cron): alerts the founder if the number of pending proposals exceeds a configurable threshold. One-hour cool-down to prevent duplicate alerts. Statistics include totals by status, average validation time, and additions in the last hour. The alert is routed through the central email facade — never via a direct SMTP connection.
  • Human validation: a dedicated dashboard page lists proposals and allows each to be validated or rejected. A validated proposal is promoted to the native skills registry.

Not to be confused: the operational detector described above is the only one that actually feeds the proposal log. A second skeleton module exists (Sunday cron 03:00 UTC) whose docstring explicitly states that it scans and logs without yet creating proposals — the detection logic is forthcoming. It is not this module that produces active proposals.

4.6 Related Automations in the Learning Loop

Nightly Consolidation (Maintenance Loop)

A nightly maintenance automation (daily execution at 04:00 UTC) consolidates learning across ten distinct features. Each feature resolves its language model from the central AI router. The absolute cross-cutting rule: no automatic promotion from the inbox to permanent or current memory — all promotion remains human-gated. No-write mode (--dry-run) is the default.

  • Repeated scar detection: compares new entries from the day against past entries via vector similarity. If at least two similar occurrences are found, a promotion draft is generated in the inbox.
  • Session compaction: analyzes the day's session logs (maximum 30, skipping those containing secrets detected by regex), extracts 3 to 5 durable facts (decision, delivery, scar, finding) and records them in the daily journal. In dry-run mode, no LLM call is made.
  • Morning summary: a digest email sent via the central email facade, only if drafts or journals were produced overnight.
  • Documentary divergence audit: checks canonical pricing tiers, scans obsolete patterns defined in a configuration file, and flags numerically dense tables that would benefit from database storage. Discrepancies are deposited as drafts in the inbox.
  • Nightly reflection: for each active agent, the 20 most salient resolved scars feed an LLM that produces 3 introspective questions and 3 abstractions, deposited as drafts. In dry-run mode, only the count of eligible agents is performed.
  • Forgetting curve: archives (without deletion) very old scars with no recorded recall, capped at 200 per run. Archiving renders them invisible to the semantic recall engine but remains reversible.
  • Project proactivity: monitors projects with active external contacts, automatically links corresponding inbound and outbound emails, and submits the reconstructed thread to an LLM that classifies the situation: to archive, action required, awaiting external response, or no action needed.
  • Internal obsolescence audit: evaluates projects inactive beyond a configurable day threshold, computes their obsolescence signals, and produces a verdict (still relevant / possibly obsolete / archiving recommended) persisted in the obsolescence log. Non-"still relevant" verdicts surface in the morning email for operator decision.
  • Persona drift audit: runs only on the 1st of each month. Compares each agent's current profile against its reference configuration, computes a drift score, persists it in history, and surfaces the top 3 most-drifted agents in the morning email.
  • Daily cost ticket: aggregates the previous day's AI costs by agent, compares against the 7-day average, and sends an email alert if the delta exceeds 20% for a given agent.

Scar Classifier

A qualification automation classifies scars that have not yet been categorized. The language model returns a structured object containing:

  • A proposed category for the error type.
  • A re-estimated severity (updated if different from the current value).
  • A confidence index and a rationale.
  • Proposed tags.

Note: the learnability flag (learnable) is not written directly to the corresponding column by this automation. It is serialized as a prefix in the rationale field in the form learnable:true|false — documented and intentional behavior.

Current Memory Hygiene

A monthly consolidation pipeline (first of the month, 00:00 UTC, with anti-overlap lock) cleans current memory in four sequential steps:

  1. Environment validation: checks that required environment variables and paths are accessible. On failure, the pipeline halts without executing subsequent steps.
  2. Dead link detection: scans the memory index file and produces a report of broken references. This step runs even in dry-run mode and may write reports to disk.
  3. Obsolete file archiving: moves files marked as decommissioned or obsolete to an off-repository archive area and removes the corresponding references from the index.
  4. Fingerprint deduplication: identifies and removes strictly identical files (SHA-256 fingerprint).

A JSON report is generated on each run, indicating for each step the status, duration, and any errors. Exit codes are:

Code Meaning
0 Pipeline completed successfully (or validation-only succeeded)
1 Environment validation failed — no steps executed
2 Fatal error in a step — pipeline interrupted
3 Dry-run mode completed — no actual changes made

Available flags: --dry-run (steps 3 and 4 bypassed or simulated, step 2 always active), --validate-only (halt after step 1).

Important: the training module documenting this pipeline (consolidation training material) is hard-read by the non-regression quiz. Deleting it without first updating the quiz would cause at least four checks to fail and would block the validation pipeline.

Current Memory Observability

A daily automation (06:00 UTC) records a 90-day time series: file count, total size, number of lines in the index, dead links, and orphaned references. Alerts are emitted to the monitoring system if configured thresholds are exceeded. Flags: --json, --dry-run.

4.7 User Model Recall (Structured Channel)

Semantic vector recall is not the only memory channel. A dedicated module loads the user model from the database — a structured profile in JSONB dimensions — and injects it into agent briefings. This recall is targeted and structured, not vectorial.

  • Available dimensions: communication style, schedules, profile, skills, history, preferences, strictly private data reserved for the primary operator.
  • Composite slice core: aggregates communication style, cognitive preferences, and appointment slots, calibrated for briefing injection (under 1,000 tokens). Never includes reserved private data.
  • API: get_profile(slice=…) returns a dictionary; format_for_briefing(…) returns text ready for injection. A command-line interface allows these slices to be queried directly, with --json and --format-briefing options.

Nightly Maintenance Loop

The nightly maintenance loop forms the meta-layer above the learning cycles: an orchestrator that runs every night, inspects documentation fidelity against the codebase, measures overall system health, mechanically repairs dead references, triggers autonomous regeneration of derived chapters, and then pushes a sanitized snapshot to synedre.com.

The step sequence executes in the following order:

  1. Initial inspection — the drift detector measures the gap between documentation and code.
  2. Coverage — the blind-spot detector identifies undocumented areas.
  3. Repair — the dead-reference fixer mechanically processes broken links.
  4. Deep regeneration — the gated deep-rewrite step rebuilds derived chapters.
  5. Proposal — a staging cycle produces drafts in the staging area.
  6. Documentation publishing — chapters pass anti-leak, anti-drift, and style gates before activation.
  7. Re-inspection — the drift detector performs a second measurement after regeneration, so that the health report reflects the actual post-regeneration state rather than the state at the start of the night.
  8. Health report — the five vital dimensions are aggregated and a care note is written.
  9. Snapshot publishing — a sanitized snapshot is pushed to synedre.com (best-effort mode).

Automatic triggering occurs every day at 05:00 UTC via the system task scheduler. Execution logs are retained in the log directory dedicated to this loop.

Loop Orchestrator

The orchestrator is the single entry point of the nightly loop. It embeds several global safeguards:

  • Database circuit breaker: if the entry corresponding to this loop in the automation configuration table indicates active = 0, execution halts silently.
  • Overlap prevention lock: a temporary lock file bearing the current PID is created at startup. If a process identified by that PID is already active, the loop stops immediately. The lock is always removed at the end of execution, even if an exception occurs in a sub-step.
  • Semantic return codes: codes 1 and 2 signal "drift present" and "critical health" respectively — these are informational statuses, not fatal failures. The only blocking non-zero exit case is an actual crash of a sub-step.
  • Daily report: at the end of the loop, an idempotent record is inserted into the morning report table, summarizing the return code of each step.

The available control options are as follows:

Option Effect
(none) Full execution in live mode
--dry-run All sub-steps operate in simulation; the DB circuit breaker is bypassed
--no-deep Skips the deep regeneration step
--deep-max N Hard cap on the number of chapters processed in depth (default: 4); actual convergence remains bounded by the time budget
--deep-budget-sec N Maximum time budget in seconds for deep regeneration (default: 4,500 s / 75 min); no new chapter is started beyond this threshold

Multi-dimensional Health Report

The health report component aggregates five vital dimensions into an idempotent snapshot per execution date, visible in the morning dashboard.

Dimension What is measured Critical threshold
Proprioception Documentation drift table ≥ 1 chapter with a dead reference
Coverage Documentation coverage table ≥ 1 uncovered unit
Debt Prioritized task backlog ≥ 1 maximum-priority task pending
Learning Materialized view of scar indicators Refresh absent for more than 3 days
Automations Cron error log + automation registry ≥ 1 script in the mothership scope unintentionally disabled

Monitored automation scope: only automations belonging to the mothership scope and marked active are taken into account. Automations belonging to a client tenant or intentionally disabled do not degrade the overall health metric.

Care note: the Nightingale agent (Health Guardian) reads the aggregated vital signs and writes a care note of at most three sentences, in English. This note is stored in the JSON field of the global health report record. If the agent is unavailable, the field remains empty without blocking the rest of the loop.

Available options: --dry-run (displays the report without writing), --no-voice (metrics only, without care note). Exit codes: 0 = nominal, 1 = warning, 2 = critical.

Sanitized Snapshot Publishing to synedre.com

This component reads in read-only mode the latest health and drift snapshots from the mothership VPS, applies an automatic sanitization pass (removal of IP addresses, internal paths, and client identifiers), then pushes a JSON snapshot to the synedre.com VPS database.

Safeguards:

  • No writes are performed on the mothership database: the connection is strictly read-only.
  • The destination table is created idempotently on first run if it does not exist.
  • Only one snapshot is retained per date (UPSERT on the date key).
  • An error during the push to the synedre.com VPS generates an error log and a return code of 2, but does not interrupt the nightly loop (best-effort mode).

The snapshot exposes overall and per-dimension health (scores, statuses, sanitized summaries), the list of drifting chapters (with their scores), and the Nightingale agent's care note. These data feed the maintenance dashboard accessible from the documentation section of synedre.com.

Options: --dry-run (displays the JSON without writing). Exit codes: 0 = nominal, 1 = mothership read error, 2 = push error to synedre.com.

Centralized AI Facade

All calls to artificial intelligence services within the system pass through a single facade. This policy ensures that any change of provider, model, or key is made in one place, without touching the business automations.

Two canonical functions are exposed:

  • Vectorization (embed): transforms a text into a floating-point vector, used for semantic search and memory recall.
  • Completion (complete): sends a mission and a context to a language model and returns the textual or JSON response.

Database-driven Routing

When the provider parameter is set to auto, the facade reads the AI routing table in the database to determine which provider and model to use according to the requested functionality. This table exposes a global fallback row as well as one row per named functionality, merged with the fallback.

Key advantage: switching provider amounts to modifying a single row in the database — no redeployment is required. The nightly loop re-reads the configuration at each execution. If the database is unreachable or empty, canonical default values take over (fail-soft mode).

Default providers and models:

Usage Supported providers Default model
Vectorization Mistral, Voyage, OpenAI mistral-embed / voyage-3 / text-embedding-3-small
Completion Anthropic (Claude), Mistral, OpenAI claude-haiku-4-5 / mistral-large-latest / gpt-4o-mini

The canonical vectorization provider is Mistral, chosen for European data sovereignty reasons.

Anthropic Completion: Two Billing Paths

The Anthropic provider has two distinct call paths, selected automatically at runtime:

  • Direct API call: if the Anthropic provider API key is present in the environment or passed as a parameter, the facade calls the REST API directly. This path is used for clients with their own Console account (separate billing). It includes retry logic on transient errors (code 429 or 5xx), consumption tracking in the database (tokens, estimated cost, per-tenant billing label), and resolution of short aliases (haiku, sonnet, opus) to full model identifiers.
  • Call via the internal agent: in the absence of an API key in the environment, the facade delegates to the Gauss agent (neutral persona), which uses the internal plan. This path is reserved for internal mothership use.

Mistral Completion

Mistral calls go through the REST API with zero temperature (deterministic outputs) and JSON mode if requested. The facade makes up to three attempts in case of a transient error (code 429 or 5xx) with increasing wait time between each attempt. Non-transient 4xx errors (invalid parameter, permanently exceeded quota, etc.) are considered permanent and do not trigger a retry. Any None return is logged to the error output.

Consumption Tracking

Each completion performed via the Anthropic direct API is traced in the database: provider, model, functional label, number of input and output tokens, estimated cost in USD. This mechanism enables per-tenant chargebacks. The database write is wrapped in a protection block: a tracking error never blocks the completion itself.

API Keys and Security

Absolute rule: no key or secret value appears in this documentation or in the versioned repository. All keys are read from environment files excluded from version control.

The facade reads the following environment variables at startup:

Provider Environment variable Usage
Mistral Mistral API key Mistral vectorization and completion
Voyage Voyage API key Voyage vectorization
OpenAI OpenAI API key OpenAI vectorization and completion
Anthropic Anthropic API key Direct API completion (client path)
Database Main database password Connection to the mothership persistence engine

Loading is performed at startup from the environment files located at the root of the repository, via os.environ.setdefault (values already present in the system environment are not overwritten).

Database Connection

The mothership persistence engine is a containerized PostgreSQL service. Python automations connect to it by specifying the correct database name and the schema dedicated to the mothership via the search_path option. The port exposed on the host machine differs from the container's internal port to avoid any collision.

All read accesses (semantic recall, vector synchronization) as well as write accesses (logs, health reports, daily reports) use the same connection string, guaranteeing a single configuration point for all automations.

Component Summary Table

This table lists all components that make up the learning and memory loop of Synedre OS, organized by functional role.

Indexing and Memory Components

Component Type Role
Note Indexer Periodic task (every 15 min) Transforms notes from the documentary brain into queryable relational entries.
Vector Synchronizer Nightly task + online trigger Computes and stores vector representations (1024 dimensions) for all sources; incremental mode at night, on-demand single recompute during a reasoning cycle.
Vector Table Table Stores semantic vectors with a cosine HNSW index for approximate nearest-neighbor search.
Indexed Notes Table Table Relational representation of the documentary Zettelkasten, populated by the note indexer.
Golden Recall Dataset Reference file Reference question-answer set used by the recall evaluation harness.
Recall Baseline Frozen reference file Frozen reference scores (recall@k, MRR, top-1) serving as a non-regression threshold.

Recall and Evaluation Components

Component Type Role
Recall Engine RAG Combines lexical, semantic, and hybrid search (reciprocal rank fusion, k=60); selects top-K results, re-ranks them, and increments their usage score.
Recall Evaluation Harness Quality assurance Measures recall@k, MRR@10, and top-1 precision against the golden dataset; blocks any regression relative to the frozen baseline.
Pre-Project Recall Event hook (triggered before creation) Executes a hybrid top-4 recall before any project structure is created; non-blocking, enriches the agent's context without interrupting the flow.
User Recall Personalization Leverages the multidimensional user model to produce contextualized briefings for agents.
User Model Table Table Stores the JSONB dimensions of each user's profile, used to personalize recall briefings.

Nightly Maintenance Components

Component Type Role
Nightly Maintenance Orchestrator Nightly task (around 5 a.m.) Coordinates the full regeneration loop: perception → coverage → repair → regeneration → proposal → documentation publishing → re-perception → care → publishing.
Health Check Diagnostic Assesses five system health dimensions and records the result in the tracking tables; also feeds the team's daily log.
Public Health Synchronizer Sanitized export Produces a sanitized snapshot (free of sensitive data) of health and drift indicators, synchronized to the public VPS.
Health Snapshot Table Table Historical record of health assessments by date and dimension (five dimensions plus overall score).
Public Snapshot Table Table Day-by-day sanitized snapshots, replicated to the public VPS for exposure on synedre.com.
Nightly Consolidation Engine Task every 4 h Consolidates ten long-term memory features; runs in simulation mode by default (no writes without explicit validation).
Monthly Memory Hygiene Monthly task Cleans the memory corpus: removes dead links, archives obsolete entries, deduplicates by content fingerprint; produces a dated consolidation report.
Memory Metrics Task every 6 h Computes memory volume and quality indicators; maintains a 90-day time series and emits alerts when thresholds are crossed.
Memory Metrics Table Table Time series of memory indicators with a 90-day retention window.

Learning and Qualification Components

Component Type Role
Post-Project Reflex Retrospective Produces a structured lesson at the end of each project and injects it into the documentary brain's inbox; uses an advanced language model by default, with automatic fallback to an alternative model.
Learning Suggestion Engine LLM engine Analyzes lessons flagged as learnable and generates improvement suggestions pending human validation.
Lesson Qualifier LLM classification Assigns each lesson an error type, severity level, confidence score, and rationale; prefixes learnable lessons to distinguish them in the queue.
Lessons Table Table Stores lessons (failures and wins), their importance, and their learnable status.
Learning Suggestions Table Table Queue of LLM-generated suggestions pending team validation.
Learning Audit Log Table Immutable trace of every suggestion that has been validated and applied.

Pattern Detection and Skill Components

Component Type Role
Emerging Skill Detector Weekly task (Sunday 4 a.m.) Analyzes recurrent iteration patterns and submits new skill proposals for human validation. Functional component: proposals are actually inserted into the database.
Proposal Monitor Hourly task Emits an alert to the founding team when the number of pending skill proposals exceeds a configurable threshold.
Pattern Detection Skeleton Weekly task (Sunday 3 a.m.) — inactive MVP Traverses and logs patterns without creating proposals. Component under development, distinct from the emerging skill detector.
Skill Proposals Table Table Queue of new skill proposals pending human validation.
Native Skills Table Table Registry of the system's active skills; target for promotion of validated proposals.
Skill Validation Interface User interface (hub dial) Allows the team to validate or reject skill proposals directly from the control hub.

Cross-Cutting Components

Component Type Role
AI Facade Abstraction Single entry point for all inference operations (embedding and completion); routes each call to the appropriate provider and model via a central routing table; supports two access paths to the primary model (direct API and CLI interface); logs every call for consumption tracking and chargeback.
AI Routing Table Table Single source of truth defining, for each feature, the embedding provider, completion provider, target model, and extended parameters in structured format.
AI Consumption Table Table Records each API call with the provider, model, metering label, input and output token counts, and estimated cost in dollars; used for per-tenant billing.
User Recall Interface Skill (user interface) Allows a user to explicitly query the system's memory through a dedicated interface.
Retrospective Interface Skill (user interface) Manually triggers the production of a post-project lesson.
Win Interface Skill (user interface) Records and highlights notable successes in the memory corpus.

Architecture note: all of these components form a closed self-improvement loop. Lessons feed suggestions, suggestions feed skills, skills improve future projects, and projects produce new lessons. The AI facade and the central routing table ensure that any change in model or provider propagates uniformly across the entire loop without modifying individual components.