KIWI
Autonomous local AI agent and community of voices living in The Grove with private memory.
The Problem
Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.
My Role
Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.
Highlights
- Community of six voices living in The Grove with local memory running on the user's machine
- Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
- Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
- DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
- Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
- Deterministic audit ledger tracking every state mutation, file movement, and memory association
Stack
Constraints
- Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
- Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
- Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.
System Architecture
Key Trade-offs
The decisions worth defending — what I chose, what I turned down, and why.
File management execution model
Chose
Immutable two-phase exact-action plans with hash verification and dry-run preview
Rejected
Direct agent tool execution with broad shell permissions
Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.
Engine-to-web communication architecture
Chose
Dedicated loopback HTTP/SSE gateway with bearer token authentication
Rejected
Monolithic single-process Next.js application
Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.
Device pairing security model
Chose
Windows DPAPI machine-bound tokens with polling architecture
Rejected
Inbound SSH or remote socket servers on client machines
Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.
What I'd Do Differently
An honest retrospective — the stuff I'd change with more time, more users, or a second pass.
- 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
- 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
- 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.
Interactive Autonomous Agent Shell
Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.
Technical Deep-Dive
Architecture, specifications, and implementation details.
P0 — Chat transport continuity
Revision 13, 2026-09-09: crash recovery for owner web chat turns (closes the reconciliation left open by revision 12). When the admin browser disconnects mid-turn, the engine still completes and persists the round in the scoped thread, but the SSE route stops at stopped() and never saves the assistant reply. Recovery: the engine exposes GET /api/chat/scoped/thread (src/grove-api.ts, read-only, validates the owner-web:v1:<sha256> identity) via ConversationManager.getTurns(chatId) (src/conversation/manager.ts, a copy of the thread, never mutates the store), and the web consumes it on GET /api/chat/history?channel=web&sessionId=... for admin accounts through reconcileOwnerChat (web/src/lib/chat-reconcile.ts; bridge ownerChatTurns in web/src/lib/owner-chat.ts). Rule: only the LAST message is repaired — if the last web message is a user with no assistant after it and the last completed engine round has the same user content, the missing reply is appended (at most one append, idempotent). An engine thread ending in a lone user means it is still generating → no reconciliation. Best-effort: an engine error is ignored/logged and the history is still served; multi-gap is deliberately rejected (appending at the tail would scramble ordering). Verification: npm run test — 101 passed (including 3 new: getTurns copy without mutation, thread endpoint identity validation + correct reply, empty/untouched thread), one optional semantic-model skip; npm run test:mind — 222 in 33 suites (2 new suites: chat-reconcile.test.ts + chat-history-reconcile.test.ts); both typechecks passed. Local changes; no production restart or deployment.
Revision 12, 2026-09-09: Telegram can no longer deliver the same update twice, and concurrent deliveries can no longer exceed the message quota. The webhook (web/src/app/api/telegram/webhook/route.ts) reads update_id and records it exactly once in telegram_webhook_updates — key (chat_id, update_id) (safe: getLinkByChatId enforces at most one active link per chat, and update_id is strictly increasing per bot), 200 without processing on duplicates, pruning rows older than 48h. The quota is consumed atomically via consumeMessageIfUnder (web/src/lib/billing.ts, a single conditional UPDATE with changes === 1), placed after the file-only filter and before appendMessage, replacing checkUsage+consumeMessage: concurrent requests neither double-charge nor overshoot the ceiling. The engine (src/telegram/run.ts) persists the update_id offset in data/telegram-owner-offset.json (via dataPath, honours KELVIN_DATA_DIR) and resumes from it after a restart, avoiding a ~24h queue replay. Crash reconciliation for web chat turns was closed in revision 13. Verification: npm run test — 97 passed, one optional semantic-model skip; npm run test:mind — 210 in 31 suites (5 new tests); both typechecks passed. Local changes; no production restart or deployment.
Revision 11, 2026-09-09: both chat routes (JSON and SSE) now accept bodies of at most 500,000 bytes and messages of at most 100,000 characters, via web/src/lib/chat-body.ts (streaming read, 400 for a missing/invalid body or empty message, 413 over the limits), and the engine rejects in chatHandler non-string or over-cap messages with 413 without contacting the manager. Verification: npm run test — 97 passed, one optional semantic-model skip; npm run test:mind — 205 in 30 suites; both typechecks passed. Local changes; no production restart or deployment.
Revision 10, 2026-09-08: the technical statements below (per-thread concurrency and coordinated deletion) are verified; anything that does not yet reach acceptance posture stays marked. This keeps scoping the owner browser transport; it is not two-Windows-system or live-provider acceptance.
Revision 10 evidence: npm run test — 96 passed, one optional semantic-model skip; npm run test:mind — 200 in 29 suites; both typechecks passed. Nine new tests landed: three for serialization/coordinated deletion in the manager and the Grove route, six for the web session-delete route. Code changes remain local; no production restart or deployment.
#Revision 10 — same-thread concurrency and coordinated deletion
ConversationManager now serializes delivery per thread: reply/replyStream wait behind any earlier turn through runSerialized, so one conversation's history can no longer be interleaved by two concurrent requests. Each thread runs at most one turn at a time; distinct threads stay independent.
Coordinated deletion now joins the web and the engine when a session is removed. The engine exposes DELETE /api/chat/scoped, which validates the owner-web:v1:<sha256> identity exactly like the corresponding POST, then forgets the thread through clearWhenIdle — it waits for any in-flight turn to settle before wiping the history, so a deletion cannot resurrect a queued turn over an emptied thread. The web, in DELETE /api/chat/session, removes the SQLite session first and then — only for owner browser sessions — asks the engine to forget the matching scoped thread under the same conversation id. Engine failure never blocks the web deletion: it is logged and ignored (the web store stays authoritative).
Contract: ownerConversationId(userId, sessionId) is the single source of truth for the thread identity at both write and delete time (the same sha256 of the account/session pair). owner-web: threads remain separate from shared channels and other accounts; approvals stay bound to that identity. The new DELETE accepts no other identity and touches no other engine thread.
Remaining P0, not closed here: full audit of background automations/plugins outside the conversation/registry path, declared plugin capabilities (closed in revision 15 — see host actions) and per-account persistent personal state. Crash reconciliation for web chat turns was closed in revision 13 (see above), distinct from the revision-12 Telegram dedup. Legacy histories are not auto-imported; thread persistence follows the existing persistConversations option; approvals do not survive restart.
Revision 9 build/browser evidence: npm run verify:web passed with 478 sources in .verify-work/web-NwBtZf/project; verification.json reports build exit 0 and no production database/restart. node scripts/verify-chat-http.mjs .verify-work/web-NwBtZf/project passed using the new scoped endpoint with a mock engine, real Edge/web HTTP/temporary SQLite. Artifacts: project/chat-probe/743dc227-e3ad-49d8-8ba3-ee9b914156a3/ within that snapshot. Two stream POSTs, zero JSON replay, two engine calls, two user rows and one assistant row. This does not test live providers or two PCs. Transient build network/TLS retries recovered; existing JOSE/Edge and scheduler-import warnings remain.
Revision 9: admin browser chat now uses engine POST /api/chat/scoped. The web server derives owner-web:v1:<sha256> from JSON [authenticatedUserId, ensureSession(...).id]; it never forwards a browser-supplied engine identity. The manager keeps these histories separate even with conversation.shared enabled; approvals stay bound to that identity. Missing/malformed scope is rejected, and an unavailable owner brain returns 503 without mind.hear. The engine echoes the scope; the web rejects mismatches without retry. Older engines lack this endpoint and cannot silently process the request in the legacy shared thread. Long-term owner memory/persona remain shared; this is not tenant isolation of the entire engine. Verification: 93 engine tests passed, one optional semantic skip; 194 mind/web tests in 28 suites and both typechecks passed. No production rollout.
Legacy histories are not automatically imported into new threads. Resuming an old UI chat keeps its SQLite display, but the new engine thread starts without the old shared transcript. Thread persistence follows the existing persistConversations option; approvals do not survive restart. Deleting web history now also clears the matching scoped engine thread for owner browser sessions (revision 10). Same-thread concurrency is serialized; coordinated deletion/export for all users and persistent personal state remain open. Roll out the engine before the web; there is no fallback to the old endpoint. Engine regressions exercise the real manager and Grove route (HTTP injection, mocked mind), without real actions. Web tests cover account/session hashing, response mismatch and coordinated deletion.
Revision 9, 2026-09-08. Implemented locally; P0 remains open. This follows the account/operator boundary of ADR-0006 and the approved requirement against competing fallback memories.
#Routing contract
| Caller | POST /api/chat | POST /api/chat/stream |
|---|---|---|
| Admin browser without Authorization | Owner engine through ownerChat | Same endpoint through ownerChatStream |
| Ordinary browser | Own embedded account brain | Own embedded account brain |
| Machine credentials | Validated account identity, existing subscription checks, own embedded brain | HTTP 401, even with an admin session cookie |
| Missing/invalid identity | Rejected | Rejected |
Both owner paths send the same message, effective locale and persona briefs to engine POST /api/chat/scoped. The SSE adapter waits for the complete engine JSON reply, then emits meta, one full-text token, and done; the route persists the assistant message and emits saved with conversation/message IDs. It does not simulate incremental typing or invoke the embedded brain for extra metadata.
Engine failure or an invalid/empty response does not switch brains. The JSON route returns HTTP 503 with ok:false, code:OWNER_CHAT_UNAVAILABLE, source:engine, outcome:unknown, sessionId and userMessageId. Failure after SSE starts emits an error frame without a success done/saved sequence. Other failures caught by the JSON generation block also return HTTP 503 instead of a fabricated successful reply.
#Browser delivery and interruptions
sendChatStream issues exactly one POST. Network failure, HTTP error, unexpected content type, server error or incomplete SSE never causes a JSON fallback request. Its parser handles split UTF-8 and CRLF frames; success requires both done and saved.assistantMessageId.
The dashboard blocks overlapping sends synchronously. A local request ID binds saved DB IDs only to that request's user/assistant rows; older unsaved error notices cannot inherit a later reply's identity. Attachments are cleared only after confirmed completion. Partial text is retained with a visible interruption notice, and history is refreshed.
Cancellation suppresses subsequent delivery/persistence when observed before the reply. A dispatched engine request may still finish; revision 13 recovers the in-flight round's reply on reload of the admin session history (tail-only reconciliation on GET /api/chat/history). Closing the browser is not a rollback or a confirmed cancellation of a host action.
#Accounting and remaining limits
The route saves the user message, links attachments and consumes quota before calling the brain. Failure does not refund quota or delete that message. An unknown result may mean the engine already performed work. There is no atomic transaction across engine execution, web history and quota, no durable request deduplication, and no exactly-once guarantee across crashes or manual resends; reconciliation recovers only the lost reply, it does not roll back a turn that was already delivered. Users should inspect history/action results before resending.
This fixes the browser transport boundary only. The legacy unscoped route retains conversation web and mind.hear fallback; new web chat uses only the manager with scoped history. Engine semantic memory/persona remain shared by the operator. Embedded account memory remains volatile; other Grove surfaces and Telegram/CLI continuity require further review. Existing histories were not merged, copied or erased. Source PDF libraries remain untouched.
#Previous evidence — revision 8
npm run typecheck:webpassed;npm run test:mind: 192 tests in 28 suites passed.- Ten route tests cover owner JSON/SSE equivalence, explicit failure, cancellation, machine/browser separation and ordinary-account routing. Seven client tests cover no retransmission and split UTF-8 delivery. Existing attachment handler tests remain green. Auth/brains are mocked in these unit tests.
npm run verify:webpassed:.verify-work/web-FI79by/project, 477 build sources; report.verify-work/web-FI79by/verification.json.node scripts/verify-chat-http.mjs .verify-work/web-FI79by/projectpassed with real Edge, web HTTP and temporary SQLite, a synthetic admin and a loopback mock engine. Two sends produced two stream POSTs, zero JSON fallback POSTs and two engine calls. DB contained two user messages and one real synthetic response; the failed call created no assistant row.- Artifacts: snapshot
chat-probe/a930c7aa-ed9a-418b-aa1c-f59f62a0aaf8/;result.jsonreportspassed:true, and the error screenshot was inspected. This is not live provider/owner-engine or two-PC acceptance. - Prior engine baseline is revision 7: 90 passed, one optional semantic test skipped. Engine sources were not changed or retested in this step. JOSE/Edge compression warnings and scheduler registration during build persist.
Sources: web/src/lib/owner-chat.ts, chat-stream-client.ts, both chat route handlers, dashboard grove-main.tsx, web/tests/chat-routing-isolation.test.ts, chat-stream-client.test.ts. Next: explicit conversation ownership and remaining internal/background execution audit, then persistent state and controlled migration checks. See state contract, verification and checkpoint.