News · · 6 min read

Building Verifiable Memory as Infrastructure

Technical look inside Cogeto, an open source, EU-hosted AI memory system built around verifiable facts, self-verifying extraction, private single-tenant architecture, source-linked memory, and provable deletion receipts.

Building Verifiable Memory as Infrastructure

AI assistants are moving from novelty into workflow infrastructure. That shift changes the engineering problem. A useful assistant can no longer behave like a stateless chat box with a few remembered preferences. It needs durable context, source awareness, policy boundaries, deletion semantics, quality controls, and an operating model that makes trust inspectable.

Cogeto is designed around that premise.

At its core, Cogeto is a private, EU-hosted AI memory system for professionals whose context is sensitive by default: consultants, lawyers, advisors, founders, and small teams. The product goal is not simply to remember. The goal is to remember responsibly: every fact must be source-linked, scoped, inspectable, correctable, and deletable. The technical architecture reflects that constraint from the first commit.

The Architectural Thesis

Most AI memory systems treat memory as accumulated text. They store conversations, chunks, embeddings, summaries, or documents, then retrieve something similar when the user asks a question. That approach works for convenience, but it is weak as infrastructure. It struggles to answer the questions that matter in professional environments:

What exactly is remembered?
Which source supports it?
Is it still true?
Who can see it?
Was it actually deleted?

Cogeto makes the memory store the primary engineering object. A memory is not a raw document. It is an extracted fact with provenance, visibility scope, quality status, sensitivity, and temporal validity. Raw sources are input material. The durable system of record is the structured fact layer.

That distinction drives the whole system design.

Modular Monolith, Dedicated Instance, Two Processes

Cogeto is implemented as a modular monolith in TypeScript, using a NestJS backend and a React and Vite frontend. The deployment unit is intentionally simple: one codebase, one Docker image, two application processes per tenant.

The app process serves synchronous surfaces: chat API, dashboard API, connectors, approval endpoints, and user-facing retrieval. The worker process owns asynchronous work: ingestion, extraction, verification, reconciliation, deletion sagas, reminders, consolidation, and execution of approved actions. The same image is used for both, with only the entrypoint changing.

This is a deliberate CTO-level tradeoff. Cogeto’s managed product is a dedicated, single-tenant instance per customer, not a shared multi-tenant SaaS seat. That makes isolation easier to reason about, but it also means every container matters. A microservice layout would multiply operational cost and complexity across the fleet. A modular monolith keeps the runtime footprint small while preserving internal boundaries.

Those boundaries are not cosmetic. The codebase is organized into bounded contexts: memory, ingestion, retrieval, agents, connectors, tasks, identity, and model-gateway. Each module exposes one public interface. Internals remain private. Cross-module communication happens through domain events via the Postgres outbox. CI enforces dependency rules so architecture does not degrade quietly over time.

Two Hard Separations

Cogeto’s architecture is governed by two separations.

The first is fast path versus slow path. The fast path answers the user. It performs retrieval and response generation. Nothing heavy happens inside a request handler. Extraction, verification, deduplication, contradiction detection, consolidation, and deletion run in background jobs.

That is how Cogeto keeps responsible memory from feeling slow. Trust machinery runs continuously, but it does not block the user’s question.

The second separation is source of truth versus index. PostgreSQL owns the authoritative state: memories, statuses, scopes, provenance, validity intervals, tasks, receipts, approvals, audit log, outbox, and job queue. Qdrant stores vectors and filterable payload copies, but it is rebuildable. If Qdrant is lost, it can be reindexed from Postgres.

This asymmetry is critical. You cannot prove deletion if your source of truth is split across opaque stores. Cogeto treats Qdrant as an index, not as memory.

PostgreSQL as the Contractual Core

The database schema is not just persistence. It is part of the trust model.

The memory table carries owner, scope, source type, source id, status, validity interval, content, embedding reference, and timestamps. Provenance is mandatory. Even user-authored facts point to a note, chat message, or source row. No orphan memories are allowed.

Quality state is constrained to a closed vocabulary: active, outdated, contradicted, uncertain, replaced, user-approved, and sensitive. These states are not loose labels added after retrieval. They are part of the domain model and enforced through the memory aggregate. Only reconciliation can set contradicted. Only the user can set user-approved. Only the deletion saga can hard-delete.

This gives Cogeto a strong invariant: every remembered fact has a source, a scope, a status, and a lifecycle.

Retrieval: Hybrid, Fused, Filtered

Cogeto retrieval combines semantic vector search in Qdrant, full-text search in Postgres, and entity matching for people, projects, and other named concepts. Results are fused using reciprocal rank fusion.

The important part is not just ranking. It is filtering.

Scope and sensitivity are hard gates inside the underlying queries. They are never applied as application-side post-filters. That matters because post-filtering is a common leak pattern: the system retrieves too broadly, then tries to clean up afterward. Cogeto’s rule is stricter. If a memory is outside the user’s scope, it should not enter the result set at all.

Quality statuses then influence ranking. Active and user-approved facts receive full weight. Uncertain and contradicted facts can be surfaced with visible warnings. Outdated facts are strongly demoted. Replaced facts are excluded by default, but remain available for temporal queries.

This makes retrieval policy-aware at the query layer, not just the UI layer.

Self-Verifying Extraction

The ingestion pipeline follows a structured sequence: ingest, chunk, extract, verify, embed and store, reconcile.

The verification step is where Cogeto becomes meaningfully different from a typical extraction pipeline. The extractor produces candidate facts. A second independent verification pass receives the candidate fact plus the source excerpt and decides whether the source supports it. Supported facts can become active. Partial or unsupported facts become uncertain and are queued for review.

This makes status earned. A fact is not active because the model sounded confident. It is active because it passed a second check against its source. The extractor and verifier use versioned prompts, and those prompts are treated as engineering artifacts with evaluation scores.

For professional memory, this matters. The cost of a wrong memory is not just a bad answer. It can become a wrong commitment, a mistaken client assumption, or a misleading summary of what happened.

Deletion as a Saga, Not a UI Button

Cogeto’s deletion model is one of its strongest architectural decisions.

Deleting a source runs a saga across PostgreSQL, Qdrant, and MinIO. First, a Postgres transaction removes derived memories and metadata, writes a pending deletion receipt, and enqueues external deletion work. Then the worker deletes vectors from Qdrant and bytes from MinIO with retries. The receipt becomes confirmed only after both stores acknowledge. A nightly sweep verifies that no orphan vectors or objects remain.

The output is a signed, hash-chained deletion receipt. That receipt is an artifact the user can inspect and export. The point is simple: the system should not merely claim that it forgot. It should produce evidence.

That is the practical meaning of verifiable memory.

Operations and Product Shape

Cogeto is designed for single-tenant EU-hosted deployment through Docker Compose. A tenant stack includes Caddy, app, worker, PostgreSQL, Qdrant, MinIO, and Zitadel. Optional redaction runs as a separate profile. Zitadel handles identity, login, SSO, organizations, roles, and audit trail, but it does not decide memory visibility. Cogeto owns scope enforcement in its backend.

Model calls go through a model-gateway seam. v1 is Mistral-first, with architecture kept open for local or alternative model endpoints later. Optional redaction can pseudonymize sensitive entities locally before external model calls, keeping the privacy posture aligned with the EU-hosted deployment model.

The product surfaces mirror the architecture. Chat is how users consume memory. The dashboard is how they govern it: inspect facts, follow provenance, edit statuses, review uncertainty, resolve contradictions, approve actions, and view deletion receipts.

Why This Architecture Matters

Cogeto’s moat is not a single algorithm. It is the combination of disciplined architecture and visible trust artifacts.

Postgres as source of truth makes recovery and deletion understandable. Qdrant as a rebuildable index keeps retrieval fast without making vectors authoritative. The outbox and Postgres-backed job queue make ingestion and deletion reliable without adding Redis or RabbitMQ to every tenant. The modular monolith keeps operational cost realistic. The worker boundary keeps heavy trust machinery off the request path. Self-verifying extraction makes memory quality explicit. Deletion receipts turn privacy promises into artifacts.

That is the difference between AI memory as a feature and AI memory as infrastructure.

Cogeto is built on a simple engineering principle: every trust claim should have something the user can inspect. If the system says it knows, show the source. If it says a fact is current, show the status. If it says something changed, show the history. If it says it forgot, show the receipt.

Most AI products ask users to trust the black box. Cogeto is being built so the box can be opened.

Read next

Lost? Good. Let’s fix it.

Whether you’re building a product or building a career, I help founders make smarter moves and engineers grow beyond just coding.