Skip to content
←Return to Projects
Case Study

KIWI

Autonomous local AI agent and community of voices living in The Grove with private memory.

The Problem

Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.

My Role

Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.

Highlights

  • Community of six voices living in The Grove with local memory running on the user's machine
  • Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
  • Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
  • DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
  • Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
  • Deterministic audit ledger tracking every state mutation, file movement, and memory association

Stack

TypeScriptNext.js 15Node.jsSQLiteDrizzle ORMDPAPITelegram Bot APIDocker

Constraints

  • Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
  • Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
  • Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.

System Architecture

Interaction Channels
Next.js 15 Web App (:3333)
Voice HUD (:4321)
Telegram Bot
Embed Bridge (:3234)
Gateway & Protocol
Private Grove API (:8431)
Device Control API
DPAPI Token Auth
Cognitive Engine ("The Grove")
Mind Core
Behavior & Persona
Memory Tiering (STM/MTM/LTM)
Skill Registry
Execution & Safety
Host Action Policy
Exact-Action Ledger
Backup & Rollback Ledger
Distributed Node Runtime
Windows Companion Daemon
Local File Executor
SQLite / Drizzle State

Key Trade-offs

The decisions worth defending — what I chose, what I turned down, and why.

File management execution model

Chose

Immutable two-phase exact-action plans with hash verification and dry-run preview

Rejected

Direct agent tool execution with broad shell permissions

Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.

Engine-to-web communication architecture

Chose

Dedicated loopback HTTP/SSE gateway with bearer token authentication

Rejected

Monolithic single-process Next.js application

Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.

Device pairing security model

Chose

Windows DPAPI machine-bound tokens with polling architecture

Rejected

Inbound SSH or remote socket servers on client machines

Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.

What I'd Do Differently

An honest retrospective — the stuff I'd change with more time, more users, or a second pass.

  1. 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
  2. 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
  3. 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.

Interactive Autonomous Agent Shell

Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.

icelegends-interactive-shell ~ [kiwi]
History: ↑ / ↓
kiwi@the-grove:~/voices$

Technical Deep-Dive

Architecture, specifications, and implementation details.

P4 — Owner-memory isolation and retraction correctness

Revision 27, 2026-09-09. First real P4 slice. Per the plan's acceptance line — "retained claims are traceable and correctable, users remain isolated, and changes can be evaluated against a baseline" — an independent Opus planning pass scoped what's actually buildable now versus what's greenfield, and found two of the four sub-claims were each real but incomplete: a security property CHECKPOINT already claimed closed, and a correction/deletion path that worked for the dashboard but leaked back through two other retrieval paths. Local code and automated verification; no production rollout, no engine restart.

#Owner-memory isolation, closed at HEAD for the first time

CHECKPOINT revision 17 states: "GET /api/kiwi/knows is engine-backed only for an admin browser without authorization; other accounts get embedded per-user payloads." That was true only in the working tree. HEAD's committed web/src/app/api/kiwi/{status,memory,knows}/route.ts all gated the owner-engine branch on isAdminUser() alone — which resolves only the NextAuth session, never the Authorization header (web/src/lib/access.ts's isAdminUser() vs. currentUserIdFrom(), which correctly prioritizes an API key over a session). A request carrying a valid admin session cookie alongside a Bearer kiwi_ak_… machine token — realistic whenever the admin's own browser also holds an agent-key-authenticated session, e.g. via the CLI or a device agent — would read the owner's private engine memory through what's meant to be a machine-scoped, per-account path. This is exactly the bypass currentUserIdFrom already guards against everywhere else (/api/devices applies the identical rule).

Fixed by resolving identity and checking the header before the admin branch is attempted at all — status.ts previously tried engineFetch first and only resolved identity in the catch block, a fail-open ordering, not merely a missing check. A second, independent instance of the same gap: knows/route.ts's DELETE/PATCH (retract/edit a memory) checked isAdminUser() alone even in the working tree — a machine credential could mutate the owner's memory through a path it could not even read. Fixed identically.

#Retraction correctness — three real leaks closed

retractMemoryById (src/mind/database/database.ts, already committed, no pending diff) zeroes importance and stamps metadata.retracted = true — but three of its four sibling read paths never excluded retracted rows:

  • getMemoriesWithEmbeddings — semantic search scores similarity*0.7 + importance*0.2 + decay*0.1; a zeroed-importance row still clears typical recall thresholds on similarity alone. A "forgotten" fact kept reaching the model even though it had vanished from the dashboard.
  • getLinkedMemories — the association graph kept resurfacing a retracted memory as a live one's "this reminds me of…" neighbor.
  • decayMemories's hard-delete sweep — retractMemoryById's own docstring promises a kept, stamped row (an audit trail, not erasure), but the sweep deletes any row with importance < 0.3, which a retracted row always has, the moment its decay_score also drifts below 0.01. Left alone, a retracted memory would be permanently destroyed by the daily tick within roughly three weeks.

All three fixed with the minimal filter (NOT_RETRACTED, or the equivalent in-JS metadata check) already used consistently by the file's other five read functions.

#Security / correctness checklist

  • Authorization symmetry. Read and write paths on owner memory now apply the identical rule (req.headers.has('authorization') || !(await isAdminUser())) — previously GET and DELETE/PATCH disagreed.
  • Fail-closed ordering. Identity resolves before the admin branch is ever attempted, so a missing/invalid credential 401s before either brain is touched.
  • No new surface. Reused engineFetch, currentUserIdFrom, isAdminUser — no new endpoints, no schema change for this half of the slice.
  • Retraction stays one-way and auditable. All three fixes exclude retracted rows from retrieval; none of them touch retractMemoryById itself, so the kept, stamped row (the audit trail) is unaffected — only which paths can still surface it to the user or the model.

#File-by-file

  • web/src/app/api/kiwi/status/route.ts, memory/route.ts, knows/route.ts — hot (large pre-existing uncommitted files, part of the same deliberately-deferred P0-era bucket as prior revisions' Telegram work). Built from HEAD's actual committed baseline in an isolated scratch copy, not the working tree, and staged via git hash-object/git update-index — confirmed via git diff --cached to be exactly this fix, nothing from the surrounding pending content.
  • src/mind/database/database.ts — clean (tracked, no pending diff at the time of this fix) — the three retraction-leak fixes, ordinary commit.
  • tests/userMemory.test.ts — clean, extended with 3 new cases.
  • web/tests/kiwi-memory-ownership.test.ts — new (7 cases).

#Deliberate non-goals for this slice

  • Per-fact source/page provenance. No storage layer anywhere — engine memories, web's embedded nature_observations, or the knowledge-search Chunk type — carries a document/page reference through to a stored memory today. Designing that schema across two unrelated stores, before any real requirement narrows the shape, would be guessing rather than building.
  • An eval harness ("changes can be evaluated against a baseline"). Nothing exists to compare against — no fixture set, no baseline run. Building one before the retraction leaks above were fixed would have baselined the buggy behavior; this is the natural next slice, not this one.
  • src/memory/* (STM/MTM/LTM tiers, MemCon, DiffCoT, commitment, firewalls). Real, deliberate design — but unreachable at runtime: ConversationManager only activates the tiered path for a MemoryLifecycle instance, and src/kelvin.ts always constructs a MemoryStoreAdapter instead, so nothing is ever written to it. Several further defects were found while reading it (a promotion step that clears 100% of short-term memory instead of the intended half, three independent MemoryStore instances silently overwriting one shared file, long-term memory permanently unrankable due to a hardcoded score, a forget skill that reports success without deleting anything) — reported, not fixed, since the subsystem's fate (repair-and-wire, or delete) is an architectural decision for the operator, not a side effect of a memory-ownership fix.
  • The SQLite tier tables (stm_entries/mtm_nodes/ltm_entries) sitting in the same pending bucket — real schema, but nothing writes to them; committing the tables without writers would be inventing state.
  • Unifying the three memory systems (engine tiered, engine flat SQLite, web embedded per-account) — a genuine architectural decision needing its own design pass.

#Verification

npm run typecheck and npm run typecheck:web: both green. Engine: 145 tests (144 passed + 1 optional semantic skip; +3). Web/mind: 41 suites / 271 tests (+7). npm run verify:imports: 496 files, 0 problems. Isolated clone (git clone --local --no-hardlinks + fresh npm ci + npx tsc --noEmit): clean except the pre-existing, unrelated stripe/index.ts/studio.ts migration gap documented in revision 26 — unchanged by this commit. [RESOLVED at rev 36] stripe/studio were deleted outright and every real importer migrated; typecheck:web has been clean on a fresh checkout since. No production rollout.

#Remaining P4 work

Per-fact provenance; the eval-against-baseline harness; the fate of src/memory/*'s dead tiered subsystem — decided at revision 68 (delete) and done at revision 69, see docs/adr/0009-three-tier-memory.md's amendment; unifying the three memory systems; the pre-existing stripe/studio typecheck gap (revision 26, still open) — resolved at rev 36.

Sources: web/src/app/api/kiwi/{status,memory,knows}/route.ts, src/mind/database/database.ts, web/src/lib/access.ts, tests/userMemory.test.ts, web/tests/kiwi-memory-ownership.test.ts, checkpoint.

~ End of Document ~