KIWI
Autonomous local AI agent and community of voices living in The Grove with private memory.
The Problem
Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.
My Role
Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.
Highlights
- Community of six voices living in The Grove with local memory running on the user's machine
- Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
- Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
- DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
- Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
- Deterministic audit ledger tracking every state mutation, file movement, and memory association
Stack
Constraints
- Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
- Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
- Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.
System Architecture
Key Trade-offs
The decisions worth defending — what I chose, what I turned down, and why.
File management execution model
Chose
Immutable two-phase exact-action plans with hash verification and dry-run preview
Rejected
Direct agent tool execution with broad shell permissions
Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.
Engine-to-web communication architecture
Chose
Dedicated loopback HTTP/SSE gateway with bearer token authentication
Rejected
Monolithic single-process Next.js application
Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.
Device pairing security model
Chose
Windows DPAPI machine-bound tokens with polling architecture
Rejected
Inbound SSH or remote socket servers on client machines
Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.
What I'd Do Differently
An honest retrospective — the stuff I'd change with more time, more users, or a second pass.
- 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
- 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
- 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.
Interactive Autonomous Agent Shell
Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.
Technical Deep-Dive
Architecture, specifications, and implementation details.
P0 — Attachment lifecycle and credential boundaries
Revision 17, 2026-09-09. Local corrections; no production migration or cleanup run.
#Revision 17 additions
POST /api/chat/files now rejects with 413 from the Content-Length header when the upload exceeds the account plan cap plus 1 MB of multipart framing, before formData() buffers anything. scripts/audit-attachment-links.mjs is a read-only preflight over message_files; scripts/repair-attachment-links.mjs (new) audits and, with --fix, repairs inside one transaction — deduplicate (message_id,file_id) pairs keeping MIN(rowid), delete orphan associations (missing message/file) and mismatched-owner associations; file storage is never touched. The production data/kelvin.db still reports tableExists:false, migrationRequired:true (the table is created at web startup; no production data changed).
#Authentication
currentUserIdFrom(req) treats any Authorization header as an explicit choice of machine authentication. Invalid, empty or revoked credentials return no identity; they cannot fall back to a browser cookie. Valid keys still select their own account. The ISO download guard applies the same no-fallback rule. The browser-only attachment API rejects Authorization headers outright. This does not certify every session-only API as machine-capable.
#Upload, listing and removal
POST /api/chat/files requires a real multipart file and an existing conversation belonging to the browser account. A foreign or nonexistent conversation returns 404 before filesystem storage or extraction. Malformed forms/files return 400. The existing free/pro/enterprise size and conversation limits remain; unknown subscription tiers use free limits.
SQLite reserves an attachment slot transactionally before disk work, preventing parallel uploads from exceeding the conversation limit. Failed storage removes the reservation and returns 503. A concurrent deletion is checked after the write so an upload does not silently recreate a deleted attachment. This is not a distributed transaction; process crashes can still leave an orphan that requires reconciliation.
Web storage names use a generated ID and a MIME-selected extension. Telegram documents/photos also use generated names; supplied filenames are display metadata, never paths. Both share uploads/chat by default, overridable with KIWI_CHAT_UPLOAD_DIR. Changing that directory does not migrate historical absolute paths; review and migrate them explicitly.
GET requires conversationId, verifies ownership and returns {files,plan} without disk paths. POST preserves existing top-level file fields and additionally returns the same object as file for the upload component. The component now supplies the selected conversation, asks before deletion and reports success only after a successful DELETE.
DELETE selects the record by owner and ID. It only unlinks a regular top-level file inside the configured upload root, rejecting redirected directories, symlinks and hard links. A missing physical file permits metadata removal; filesystem/path failures return 409 and retain metadata. Successful metadata deletion cascades to message_files. Checks are not certified against a malicious concurrent local process replacing filesystem paths.
#Expiration and scheduled cleanup
Confirmed defect: Telegram cleanup scanned the shared directory and removed every file older than 24 hours by filesystem time, including web uploads with 30-day metadata expiry.
Cleanup now selects at most 200 explicitly expired chat_files records per pass, also requiring a minimum record age (default 24 hours). It removes managed bytes and metadata; failures retain metadata for retry. It never scans and deletes unknown files. Legacy records without expires_at and untracked PDFs remain untouched. New Telegram attachments, like web uploads, receive 30-day expiry. The existing startup/six-hour scheduling remains.
Expired attachments are excluded from GET and from new chat prompt attachment resolution even before cleanup runs; they no longer consume the active upload quota. Expiration does not retroactively remove text already copied into conversation history or memory. That requires P4 provenance/forgetting work.
#Evidence and remaining work
web/tests/chat-file-lifecycle.test.ts uses real temporary SQLite and files: owner validation, multipart/auth rejection, response compatibility, parallel quota, physical deletion/cascades, corrupt paths, missing files, junction rejection, retention and Telegram filename handling. Extraction, auth and Telegram transport are simulated. access-identity.test.ts, iso-credential-boundary.test.ts and both chat-handler attachment suites cover credential fallback and expired attachments.
Remaining: production path/association audit and backups before rollout; multipart streaming size enforcement before parsing; process-crash reconciliation; stronger native filesystem race protection; equivalent concurrent Telegram quota enforcement; source-linked forgetting/export. No operation here modifies Mind memory fundamental/ or the source PDF library.