Chapters

DOC-05 / Technical reference · Chapter 05

Automations, crons & runs

Describes the non-conversational execution layer of Synedre OS: the 209 Python facades, their cron wrapper, the un composant interne registry, and the boundary between run doctrine and agent unit.

Automations, crons & runs

This page describes the non-conversational execution layer of Synedre OS: the Python facades, their cron wrapping, the automation registry, the dual scheduling system (Nitro scheduler + Linux crontab safety net), and the boundary between the run doctrine (a driven unit of work) and the agent execution unit (a delegated task). Intended audience: engineers taking over the harness.

Security note: the production crontab file may contain secrets in plain text (database passwords, authentication tokens). They are not reproduced here. This hardening debt — inline secrets in the crontab — is flagged at the end of this page.

1. Mental model: agent (thinks) vs. automation (executes)

The harness separates two registers:

Agent Automation
Role Thinks, decides, delegates (ReAct loop) Executes a deterministic routine
Substrate LLM persona (Claude / Mistral) Python script in the repository
Storage Agents table (persona registry) Automation registry (dedicated table)
Triggering Orchestrator spawn, task tool, agent task unit Cron, CLI, Nitro scheduler
Link between the two A join table links each automation to its owning agent; an agent can spawn an automation or a delegated task.

An automation does not "reason": it may call an LLM (content generation, classification) but its control flow is hard-coded. Reasoning lives in the personas and the Atlas orchestrator; repeatable execution lives in the automations.

   ALEX / EMAIL / CRON
          │ trigger
          ▼
   ┌──────────────┐  delegates  ┌──────────────┐  spawn   ┌──────────────┐
   │  Atlas (run) │ ──────────► │  agent task  │ ───────► │ LLM CLI /    │
   │  (driven run)│             │  (delegated) │          │ Python script│
   └──────────────┘             └──────────────┘          └──────────────┘
          ▲                                                     │
          │                               cron/Nitro ──────────┘
          │                               (scheduled automation)
   /hub/runs                           automation registry + automation logs

2. Python facades

The repository contains several hundred Python files. Not all of them are scheduled automations: the majority are facades (single entry point for a capability), some being shared libraries, manual tools, or one-off scripts.

The canonical classification registry lives in the database in the automation registry, not in the files themselves. Two axes structure each entry:

  • kind — technical type:
    • recurring — scheduled (cron / Nitro)
    • oneshot — triggered on demand
    • tool — tool invoked by an agent or a skill
    • lib — shared library, not standalone-executable
    • meta — meta-tooling (e.g. the cron wrapper itself)
  • caste — owning agent group. Observed castes: Vigies (audits/QA), Scribes (writing), Oracles (monitoring/reporting), Horlogers (infra/session/backup), Bâtisseurs (build/provisioning), Tisserands (SEO/interlinking), plus technical castes (execution, library, veille). ⚠ Debt: casing is inconsistent across entries — normalisation to be planned.

Functional taxonomy of facade families

Family Role Notes
Atlas orchestrator Inbox → intent → spawn pipeline; orchestrator health and monitoring
Audits Drift detection (schema, SEO, accessibility, security, keyword cannibalisation) Non-zero exit = audit findings, not a crash (see SOFT_FAIL_SCRIPTS §3)
Backups Database and file dump to object storage; monthly restore test
Brainstorm Asynchronous idea-generation jobs (loop worker + punctual cron safety net)
Blog / SEO Content generation and hygiene, technical SEO Internal interlinking engine is archived, no longer active
Email / inbox Mail reading and writing; single client-facing send facade Client send = facade only (no direct SMTP access from agents)
Browser Browser automation (residential headful worker, screenshots, QA)
Banking / billing Bank synchronisation, recurring billing, payment reminders
Brand monitoring Brand surveillance, technology watch, Google reviews Newsletter engine is archived, no longer active
Memory / learning Vector indexing (RAG pgvector), consolidation, post-mortems, pattern detection
SRE / guardrails Cost monitoring, runaway detection, alerting, production write guardrail
Documentation self-maintenance Doc↔code mirroring, blind-spot coverage, deterministic dead-path repair, deep regeneration, publication to synedre.com See §7
Autonomy Daily workstream seeder in automatic mode; bootstraps the agent task pipeline See §9
Libraries / infra Shared building blocks (DB connection, logger, environment variables, AI provider, log rotation, cron wrapper) — not scheduled

3. The cron wrapper — supervisory watchdog

Every automation scheduled via crontab is launched through a supervisory wrapper. This component is the stability guardian of the cron layer: it shields each script against error loops, attempts automatic repairs, and logs every incident to the database.

A typical invocation looks like:

*/2 * * * *  <wrapper-path> <script-name> >> <log-file> 2>&1

The wrapper's responsibilities are as follows:

  1. Environment variable loading: the wrapper reads the .env and .env.host files at startup and injects the variables into the subprocess environment without overwriting values that are already set.
  2. Circuit breaker: before each launch, the wrapper queries the error log in the database. If a script exceeds 10 consecutive failures, it is automatically disabled (the wrapper exits without executing the script). Only a single line is logged at the moment the threshold is crossed — an anti-spam measure that prevents hundreds of repetitive lines per day.
  3. Supervised execution: the script is launched as a subprocess with a default timeout of 300 seconds. A list of long-running scripts (complex LLM spawns, etc.) benefits from an extended timeout of up to 1,800 seconds.
  4. Self-repair: on error, the wrapper analyses standard error output, classifies the incident, and attempts an automatic fix before re-running:
    • Missing import → injection of the import from a known list
    • Log permission error → redirection to an accessible directory
    • Directory not found → directory creation (sandboxed under the repository root)
    • Absent Python module → installation via pip into the virtual environment
    • Log variable name conflict → automatic renaming of the conflicting alias
    • Network error → 3 retries with exponential backoff (2 s, 5 s, 10 s)
  5. Soft-fail for audits: audit scripts are declared in a special list. For these, a non-zero exit code means findings were detected, not a crash. The consecutive error counter is reset and the exit code is propagated to downstream consumers.
  6. Database logging: each incident is recorded in the cron error log table (append-only table) with the following fields: error type, traceback, automatic-fix indicator, description of the applied fix, retry success, consecutive failure count, disabled status.

Caution — no lock: the wrapper itself does not implement an exclusive lock (flock). A few crons add a lock manually (notably the brainstorm workers and certain punctual extractors). The majority of wrapper entries have no overlap protection: safety relies solely on the timeout and scheduling frequency. This is an architectural debt to be addressed.

The database connection used by the wrapper goes through the repository's shared connection facade, which points to the mothership's primary database. The MariaDB connection constants present in the code are dead legacy (removed during a previous infrastructure workstream) and can be ignored.

4. The Run System

4.1 The Run — Scoped Execution Unit

A run designates an execution driven by the central orchestrator over a defined scope (mother-ship or client). At startup, the scope automatically loads its context from the database: infrastructure, contact, inbox.

The runs table contains the following columns: identifier, source, trigger, scope, title, status, type and reference identifier, start and end timestamps.

Two triggers are active in production:

Trigger Source Entry point
email Atlas inbox Forwarded to the orchestrator's Atlas address, classified by intent (run, question, workstream, noise)
chat Console Scoped console accessible from the runs dashboard (scope selection + Atlas conversation)

A third scheduled trigger (cron) is documented in doctrine but is not yet active in the database.

The run management interface is accessible from the orchestrator's main dashboard, which exposes the list of runs and a detailed view per item.

4.2 The Agent Execution Task — Delegated Unit

The agent execution task is not a run. It is the unit of work delegated to an agent by the orchestrator. It lives under the tasks dashboard and under workstreams. A run may spawn an agent execution task when the orchestrator delegates.

The execution tasks table (approximately 209 rows) contains the following key columns: agent identifier, title, prompt, template, scope, exit criteria, status, creator, output log, exit code, tokens consumed, cost in USD, model used.

Observed statuses:

  • Completed: 182 tasks
  • Failed: 23 tasks
  • Cancelled: 4 tasks
  • Transient states: pending, running

Execution templates: code (208 tasks), audit (1 task).

Observed creation origins: automatic task executor chain (121), manual operator trigger (37), automated workstream launch API (29), automatic work chain (22).

Executor: a worker scheduled at one-minute intervals consumes pending tasks. Its logic: it selects tasks with pending status in age order, marks them as running, spawns the subprocess according to the template, streams standard output to the log, then marks them as completed or failed.

Two execution modes:

  • Simulation mode (default): no real spawn, placeholder recorded.
  • Live mode: spawns an inference process. Granted permissions vary by template:
    • research template: read-only (files, search, revision history).
    • audit template: read + report write.
    • code template: extended permissions, sandboxed to the declared scope directory.

4.3 The Automaton Execution Log

The automaton execution log (approximately 124,000 rows, continuously growing, append-only mode) verbosely traces each automaton execution. Columns: automaton name, environment, result, duration in seconds, step count, error count, warning count, counters, step detail, errors, warnings, result detail, context.

This log is distinct from the wrapper crash registry, which records only envelope-level (wrapper) failures — not nominal executions.

Component relationships:

Orchestrator run (scoped)
   └─ spawns ─► Agent execution task ──log──► output log (output_log)

Automaton registry ──scheduled execution──► Verbose automaton log
                                        └─► Wrapper crash registry (crashes only)

5. Scheduling: Two Parallel Schedulers

5.1 Nitro Application Scheduler (Mother-Ship Layer)

The mother-ship application exposes a task scheduler integrated into the application framework (experimental feature enabled in the configuration). Tasks belonging to the audit family are defined in a dedicated module automatically merged by the framework — they do not reside in the root tasks directory.

History: this scheduler was inactive for approximately three days in May 2026, then restored as part of a reactivation workstream. Tasks have been active since then, provided that a container rebuild is triggered after any configuration change (the container must be rebuilt to reflect changes).

Active tasks:

Role Cron frequency Guard
Email queue processing Every 2 min
Availability monitoring Every 15 min Internal environment only
Dictionary watch Every 30 min Internal environment only
Dependency watch Daily at 2:00 Internal environment only
Daily digest Daily at 8:00 Internal environment only
SSL monitoring Daily at 9:00 Internal environment only
Brand watch Daily at 12:00 Internal environment only

The internal environment only guard short-circuits the task if an activation variable is not set in the container environment (passed via the host environment file).

Still-disabled tasks: inbox synchronisation and client email synchronisation. The current IMAP client monopolises the Node event loop and causes cascading database timeouts; reactivation is contingent on the delivery of a non-blocking IMAP client (workstream in progress, phase B).

Tasks present in the codebase but intentionally unscheduled: blog hygiene, page audit, sitemap watch, Synedre watch, fact checking, schema watch.

5.2 System Crontab Safety Net

The server's system crontab carries the bulk of the scheduling load and constitutes a safety net independent of the application scheduler. Out of 175 total lines (state as of 7 June 2026), 55 are active. Commented lines carry archaeology markers (temporary freeze, explicit deactivation) — the crontab serves as an operational logbook.

Active families:

  • Via monitoring wrapper: monitoring (every 2 min), backup (4h), automaton audit (3h), backup audit (4h30), bank synchronisation (6h15), recurring billing (6h30), Atlas inbox polling (every min), orchestration spawn (every 5 min with delay), inbox synchronisation (every min), nightly documentation maintenance loop (4h), documentation drift detection (every 6h), automatic documentation correction (every 6h, +10 min), nightly maintenance loop (5h), autonomy loop (5h), embeddings synchronisation (5h40).
    These scripts benefit from centralised wrapper error logging and the self-repair mechanism.
  • Outside wrapper, Python modules: memory indexing (every 15 min), live-mode agent execution task worker (every min), skill indexing (every 15 min), deadlock detector (every 15 min), SRE alerts (every 30 min), cost alerts (every 6 min), drift detector (every 15 min), negotiation event extractor with flock locking (every 2 min), lead analysis (7:30 and 19:30), KPI refresh (5:15), Google review synchronisation (1:20), external document review processing (every 30 min).
  • Outside wrapper, direct Python scripts: document publication with limit (3:30), backup restoration test (5h on the 1st of the month), log rotation (6h), memory metrics (6h), session indexing (every hour), skill proposal monitoring (every hour), skill proposal detection (4h on Sundays), pattern detection (3h on Sundays), daily reaction digest (3:30).
    ⚠️ Warning: these scripts run without a monitoring wrapper. They do not benefit from centralised wrapper error logging or the self-repair mechanism.
  • Brainstorm worker: launched at server startup in persistent loop mode (detached process); a safety net every 2 min restarts the process if absent; a safety net every 3 min with flock locking catches missed one-off executions.
  • Shell and Node scripts: fleet scan (4:30), TypeScript dependency analysis (4:05), primary database backup to S3 object storage (3:30), file backup to S3 (3h), client VPS database backups to S3 at staggered times — all with notification fallback on failure. Monthly restoration test (1h on the 1st of the month), memory synchronisation (every 30 min), workstream lock cleanup (every 5 min), orchestrator lock watchdog (every 15 min).
  • Direct HTTP calls: mail queue drain every minute via POST to the local API (redundancy with the corresponding Nitro task); session replay synchronisation every 5 minutes.
    ⚠️ P0 Debt: one of these crontab lines exposes an authentication token in plaintext in the command. This token must be removed from the crontab line and loaded from an environment variable or a secured file — corrective action to be scheduled.
  • Memory consolidation: monthly consolidation script (midnight on the 1st of the month, with 600 s flock locking).
  • OSS quarantine: purge of quarantined packages according to a defined calendar window.

Existing scripts removed from the crontab (active facades, unscheduled): founder notification, sector watch — no active or commented line in the current state of the crontab.

⚠️ Registry ↔ Scheduling reconciliation (audit to produce): the canonical automaton registry is the authoritative classification source, but no automated mapping currently links each recurring-type automaton to its crontab line or Nitro task. It is impossible to determine how many automata are orphaned (registered as recurring but unscheduled). An audit is required: join between the automaton registry (filtered on recurring type), the active lines of the system crontab, and the tasks declared in the application scheduler configuration.

The browser-worker: distributed automaton outside the datacenter

The web navigation subsystem is the only platform component whose execution physically leaves the datacenter. Two distinct reasons justify this exit, and therefore two egress topologies that must not be confused.

Topology Component Where the browser runs What it handles Mode
Residential proxy Browser agent (VPS side) Headless browser on the mothership VPS IP reputation Headless + stealth profile
Remote headful worker Browser worker (residential machine side) Headful Chrome on the residential host machine Browser fingerprint Visible window, direct residential IP

6.1 Why a residential headful browser rather than a headless browser on a VPS

A residential proxy tunnel routes traffic from the VPS through a residential IP address. This is sufficient for sites that discriminate solely on datacenter IP reputation. A guard mechanism verifies before each launch that the proxy is active and that the effective egress IP address is not that of the datacenter — any accidental egress through the datacenter is blocked.

However, certain third-party protection systems (notably managed challenges of the Turnstile type) do not evaluate the IP: they analyse the browser fingerprint — JavaScript signals revealing automation (navigator.webdriver, absence of plugins, headless markers, WebRTC, etc.). A headless browser retains a bot fingerprint even behind a clean residential IP.

Doctrinal consequence: classify the protection type before coding the flow. If the protection targets the browser fingerprint, a headful browser on a residential machine is mandatory; stealth headless mode does not pass.

Hence the headful worker: a real Chrome instance (visible window, non-headless mode) runs permanently on a residential host machine that is always on, with a direct residential IP. No proxy is needed on the worker side: the correct IP is already in use, and Turnstile-type challenges pass naturally.

6.2 Job queue and lifecycle of a browser job

The browser job queue is stored in the database following the same enqueue / claim / finish / get pattern as the other platform workers. Each entry carries: a job identifier, a type (kind), a JSON payload, a status (queued, running, done, failed), a JSON result, an error message, the identifier of the executing machine, the retry count, and start and end timestamps. Indexes guarantee the performance of ordered claiming.

Inversion of control is the central principle: the mothership VPS has no inbound access to the residential machine (NAT, dynamic IP). It is the remote worker that actively polls the VPS via an outbound connection. The complete cycle is as follows:

  1. Enqueue (VPS) — An agent or a skill deposits a job into the queue. Only job types appearing on a strict whitelist are accepted; the JSON payload is validated before insertion. SQL injection is prevented by a random dollar-quoting mechanism.
  2. Atomic claim (worker → VPS) — Every 5 seconds, the worker queries the VPS via outbound SSH to claim the oldest job in queued status. The claim uses a transactional lock (FOR UPDATE SKIP LOCKED): two concurrent workers cannot claim the same job.
  3. Dispatch (worker, headful) — The worker routes the job to the corresponding business handler according to its type. The payload contains only business parameters (limits, quantities); no dynamic code evaluation is performed.
  4. 2FA code retrieval (worker → VPS → worker) — For certain flows requiring two-factor authentication, the code never arrives directly on the residential machine. The worker calls back the VPS via SSH so that the VPS reads the code from the server-side mailbox and returns it in plaintext on standard output. The 2FA code is never written to logs.
  5. Finish (worker → VPS) — Once the job is complete, the worker transmits the result and any error as a single JSON object passed on the standard input of the SSH command (never as a shell argument, to avoid breakage on special characters). The VPS updates the status, the result, and the end timestamp.
  6. Worker self-update — In loop mode, when the worker is idle (never mid-job), it periodically checks whether the repository has been updated. If an update is detected, it restarts automatically via process replacement (os.execv), ensuring that the residential machine always runs the latest code without manual intervention.

6.3 The SSH command portal (defence in depth should the key ever leak)

The SSH key of the residential machine is registered on the VPS side with a strict restriction: it does not open a free shell. Instead, each SSH connection is intercepted by a dedicated command portal, configured directly in the SSH authorization file.

This portal reads the requested SSH command, tokenises it safely (without ever passing through a shell interpreter), and only permits execution if:

  • the command prefix matches exactly the job queue management module;
  • the subcommand belongs to the authorised set (--claim, --finish, --get, --enqueue, 2FA code retrieval);
  • each parameter value matches a strict predefined pattern (numeric identifier, done|failed status, bounded alphanumeric job type, etc.).

Any unknown token or non-conforming value results in an immediate rejection with logging. The worker never prefixes its SSH commands with a directory change: the portal manages the working directory itself. In the worst case of a compromised key, the attacker can at most pollute the job queue — they cannot execute arbitrary code on the VPS.

Personal data and screenshots. The results of certain jobs (names, message excerpts) are stored in the private database as structured JSON and are never written in plaintext to logs — only a counter is recorded. Debug screenshots are purged at the end of the run by the relevant business module, not by the worker itself. A screenshot retention mode exists but is reserved for local debugging. Structured retrieval of screenshots is the subject of an ongoing workstream.

⚠️ Known latent bug: the expiry timeout for browser jobs is not currently enforced — a job whose browser hangs may remain in running status indefinitely. A fix is pending confirmation.

Documentation Self-Maintenance Loop

Since a recent project, the harness includes a coherent subsystem that measures the gap between its own documentation and its code, then corrects it autonomously using validation gates (gates). This subsystem constitutes the most recent and most intricate layer of automation.

Pipeline Architecture

The nightly loop is structured around a central orchestrator that triggers every morning at 5:00 AM and chains eight steps in sequence:

  1. Perceive — the drift detector inspects the gap between documentation and code.
  2. Cover — the blind-spot detector checks documentation completeness.
  3. Repair — the deterministic fixer resolves dead references.
  4. Assess — the health report aggregates all dimensions.
  5. Regenerate — the deep regenerator rewrites in depth (with gate).
  6. Propose — the staging module prepares chapters and produces a brief.
  7. Publish-doc — the publication module validates and activates chapters.
  8. Publish — the public synchronization module pushes a sanitized snapshot to the public site.

Additionally, the drift detector and the mechanical fixer each run on their own independent schedule (every six hours), without depending on the orchestrator. An external review module runs every thirty minutes.

The Drift Detector (Phase 0 — Perceive)

This component runs every six hours in strict read-only mode: it never writes to code or documentation, but records its observations solely in the database.

It measures three types of drift:

  • Stale doc — the referenced source file was modified after the document was written: the documentation describes an outdated body.
  • Dead reference — the document cites a file that no longer exists in the repository.
  • Delayed publication — the public chapter differs from the internal document (content divergence detected by fingerprint).

The component is idempotent: it produces only one observation per chapter per day. It returns either a no-drift signal or a drift-detected signal.

The Blind-Spot Detector (Phase 0bis — Coverage)

Launched immediately after the drift detector, this component checks documentation completeness — whereas the drift detector checks fidelity. The question it poses is: "Does any part of the harness go entirely unmentioned in the documentation?"

The method relies on a set difference:

  • Documentable body — the set of all automations and data structures in the harness.
  • Current coverage — those mentioned in at least one internal documentation chapter.
  • Blind spots — the difference: components present but never documented, grouped by family.

An isolated orphan is not a concern. However, multiple orphans from the same family constitute a candidate for a new section or chapter, created by the deep regenerator in creation mode. This component is also read-only and its "coverage" dimension is consolidated into the health report.

The Deterministic Dead-Reference Fixer (Phase 1bis — Repair)

This component bridges the gap between the drift detector — which diagnoses dead references — and the LLM regenerator — which corrects prose but is unaware that a file has been moved. Resolution is deterministic, with no possibility of hallucination.

For each dead reference found in the internal documentation, the component searches for the actual location by base filename:

  • Single result (certainty) → the reference is corrected in place.
  • No result (component deleted) → flagged, reported, and the founder is notified for human review.
  • Multiple results (ambiguity) → flagged and reported, no automatic rewrite under any circumstances.

Safety constraints: this component only touches internal documentation files — never code, never published chapters. It is idempotent and produces a single, reversible commit per run.

Important distinction: this fixer handles paths only in a deterministic manner and is triggered by the orchestrator. The mechanical fixer described below attempts other types of structural corrections and runs on its own independent schedule.

The Health Report (Phase 2 — Assess)

This component aggregates four dimensions without modifying any of them:

Dimension What is measured
Proprioception Documentation ↔ code gap (drift detected by the drift detector)
Debt Backlog of open priority tasks
Learning Signal derived from accumulated scars
Automations Cron health (detected errors)

The health report produces an idempotent snapshot per dimension per day, as well as an item for the daily summary. It returns a signal: OK, warning, or critical state.

The Mechanical Fixer (Independent of the Orchestrator)

This component runs every six hours on its own schedule, offset by ten minutes relative to the drift detector. It only touches internal documentation files — never code, never published chapters.

Safety rule: a dead reference is corrected only if its base filename matches exactly one file tracked by the repository. Zero candidates (component deleted) or two or more candidates (ambiguity) → submission for human validation, with no automatic write under any circumstances.

The component is reversible (a single commit per run, undoable in one operation), capped by a configurable maximum, and equipped with a global circuit breaker. It operates in simulation mode by default; active mode must be explicitly requested.

The Deep Regenerator (Phase 4 — Regenerate)

This component is called by the orchestrator with a maximum time budget (default: 4,500 seconds). It launches a headless Claude process that: (a) rewrites the internal document by reading the actual code, (b) directly produces the bilingual FR+EN public version with its metadata in a temporary workspace.

The internal document is committed, then an anti-leak gate (multi-layer regex verification) is applied before any public staging. This gate is independent of the orchestrator.

The Staging and Brief Module (Phase 5 — Propose)

Launched by the orchestrator after deep regeneration, this component re-stages chapters whose internal document is ahead of the published version, triggers unit publication, and places sanitized bilingual chapters into the publication queue. An independent anti-leak gate blocks any chapter containing a detected leak. The component also produces a regeneration brief for ambiguous cases requiring human attention.

The Nightly Orchestrator (5:00 AM Cron)

The orchestrator triggers the eight steps in sequence, under several global safeguards:

  • Overlap lock — only one instance at a time.
  • Database circuit breaker — if the kill-switch is activated for this automation, the tick stops immediately.
  • Configurable time budget for the deep regenerator; a broader external timeout encompasses the entire run.
  • Step isolation — if a step fails, the tick continues and records the failure.

At the end of the run, the orchestrator calls the public synchronization module to push a sanitized snapshot to the public site.

Typical invocation (with simulation or without deep regeneration):

python3 [orchestrateur] [--dry-run] [--no-deep] [--deep-budget-sec N]

Public Synchronization

Called at the end of the tick, this component reads loop data in read-only mode, sanitizes it (zero client names, IP addresses, secrets, internal paths), then pushes a JSON snapshot to the public site database via idempotent upsert. This snapshot feeds the maintenance dashboard accessible from the documentation section of the site.

Chapter Publication to the Public Site

This component is invoked via two paths:

  • Autonomous scheduling every night at 3:30 AM, with automatic activation.
  • Orchestrator invocation at the "publish-doc" step, with the same parameters — making the orchestrator self-sufficient without depending on the separate cron.

The component detects chapters whose internal document fingerprint diverges from that of the published chapter, re-stages them into the publication queue (secure transport to the public site database), with a multi-layer regex anti-leak pre-check.

Activation logic: a deliberate human gesture (--activate) or automatic activation (--auto-activate) can publish a pending chapter. In automatic mode, chapters passing all machine gates — including the anti-guru verification defined in public register §VII — are published without human intervention. Rejected chapters remain in the queue and generate a notification to the founder.

External Review — Every 30 Minutes

This component runs every thirty minutes independently of the orchestrator. It dequeues reviews submitted by a human operator (response from an external model), evaluates them via a sandboxed Claude process with anti-injection delimiters, then selectively parses the verdict: overall validity and injection-attempt detection.

If the review is valid, the component emits a signal only (a note in the daily summary) without triggering an inline regeneration. Regeneration is consumed during the next orchestrator cycle, following a loose coupling pattern that avoids timeouts and wasted computation.

The Autonomy Tick — Pipeline Bootstrap

This component runs daily at 5:00 AM (simultaneously but independently of the documentation orchestrator). Its doctrine: simulation by default — it observes without seeding anything; active mode must be explicitly requested.

Its role is to address the cold-start of the autonomy pipeline. The task engine chains itself within an active job, but nothing bootstraps the pipeline: a job in automatic mode whose tasks are all pending and whose no run is active remains dormant. This tick creates a pending run for the next eligible task of each active job with no current run.

Safeguards:

  • Cost ceiling — if the cumulative cost of a project reaches 100% of its budget, it is frozen and no run is seeded.
  • Dual circuit breaker — two environment variables allow the component to be fully halted without any action.
  • Deployment — signaled only, never executed within the tick; production deployment remains a deliberate human gesture.

Constitutional rule: the tick may proceed up to pre-production (./deploy) but never calls production deployment (./ship). The production deployment decision remains an explicit human gesture.

Safeguards & Operations

This section lists the protection, monitoring, and operational maintenance mechanisms that govern the execution of automations on the mother ship.

Active Protection Mechanisms

  • Automatic circuit breaker: a supervision component wraps each scheduled automation. After ten consecutive crashes, it automatically disables the automation in question and records the state in the scheduling error tracking table, thereby preventing any infinite failure loop.
  • Daily audit: every night, an audit automation re-reads the scheduling error logs as well as the automation execution logs in order to detect any persistent anomaly.
  • Cost and runaway monitoring: three watchdog automations run continuously — one every six hours to alert on cost drift, another every fifteen minutes to detect runaway executions, and a third every thirty minutes for service reliability alerts.
  • Backups and restore testing: nightly dumps are pushed to remote object storage (mother-ship database, files, remote client databases). A monthly automation replays a full restore test to validate backup consistency.
  • Automatic unblocking: every fifteen minutes, an automation detects stuck task executions and resubmits them, up to a maximum of four attempts. An emergency switch allows this mechanism to be disabled without modifying the scheduler.
  • Documentation maintenance loop: a set of three automations handles documentation drift monitoring (every six hours), automatic correction of detected anomalies (offset by ten minutes on the same cycle), and a nightly maintenance dial (each day at 5 a.m.). These mechanisms are described in detail in the section dedicated to documentation regeneration.

Reported Technical Debt

  • ✅ Audit automation removed without a replacement (resolved 2026-06-06): a URL-leak audit automation had been removed from the repository without a successor being put in place. The corresponding scheduling entry was deleted from the scheduler. Only a folder of stale logs remains. No URL-leak audit facade currently exists; it will need to be recreated via the standard scheduling wrapper if the need arises again.
  • ⚠️ Authentication token in plaintext in the scheduler: a scheduler entry embeds a bearer token directly in the command line, bypassing the secure environment file. Migration to environment-variable-based retrieval is required — the clean pattern is already applied on other scheduler entries (reading the database password from the .env file).
  • ⚠️ Duplicate email queue execution: two scheduling entries independently trigger the email send queue processing every minute. The mechanism is idempotent thanks to a LIMIT clause on the queue side, but the redundancy still needs to be cleaned up to avoid any operational confusion.
  • ⚠️ Case inconsistency on the automation classification field: the field designating the execution type shows non-uniform capitalisation (execution vs Execution) in the automations table. Normalisation is to be scheduled.
  • ⚠️ No generic lock in the scheduling wrapper: the wrapper that encapsulates scheduled automations does not acquire a system-level lock by default. Execution overlaps remain possible on slow jobs that do not handle their own mutual exclusion.
  • ⚠️ Autonomous seeding automation still in observation mode: the autonomous seeding automation is currently running in dry-run mode — it observes and logs without performing any real actions. Active mode is not yet wired into the scheduler. Additionally, a safeguard timeout on browser jobs is defined in the code but not enforced at runtime: a browser job could theoretically remain suspended indefinitely. A fix is to be confirmed.