Skip to content
←Return to Projects
Case Study

KIWI

Autonomous local AI agent and community of voices living in The Grove with private memory.

The Problem

Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.

My Role

Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.

Highlights

  • Community of six voices living in The Grove with local memory running on the user's machine
  • Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
  • Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
  • DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
  • Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
  • Deterministic audit ledger tracking every state mutation, file movement, and memory association

Stack

TypeScriptNext.js 15Node.jsSQLiteDrizzle ORMDPAPITelegram Bot APIDocker

Constraints

  • Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
  • Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
  • Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.

System Architecture

Interaction Channels
Next.js 15 Web App (:3333)
Voice HUD (:4321)
Telegram Bot
Embed Bridge (:3234)
Gateway & Protocol
Private Grove API (:8431)
Device Control API
DPAPI Token Auth
Cognitive Engine ("The Grove")
Mind Core
Behavior & Persona
Memory Tiering (STM/MTM/LTM)
Skill Registry
Execution & Safety
Host Action Policy
Exact-Action Ledger
Backup & Rollback Ledger
Distributed Node Runtime
Windows Companion Daemon
Local File Executor
SQLite / Drizzle State

Key Trade-offs

The decisions worth defending — what I chose, what I turned down, and why.

File management execution model

Chose

Immutable two-phase exact-action plans with hash verification and dry-run preview

Rejected

Direct agent tool execution with broad shell permissions

Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.

Engine-to-web communication architecture

Chose

Dedicated loopback HTTP/SSE gateway with bearer token authentication

Rejected

Monolithic single-process Next.js application

Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.

Device pairing security model

Chose

Windows DPAPI machine-bound tokens with polling architecture

Rejected

Inbound SSH or remote socket servers on client machines

Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.

What I'd Do Differently

An honest retrospective — the stuff I'd change with more time, more users, or a second pass.

  1. 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
  2. 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
  3. 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.

Interactive Autonomous Agent Shell

Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.

icelegends-interactive-shell ~ [kiwi]
History: ↑ / ↓
kiwi@the-grove:~/voices$

Technical Deep-Dive

Architecture, specifications, and implementation details.

P1 — Windows device pilot

Revision 20 (2026-09-09): device_tasks/device_pairings rows were only ever state-transitioned, never removed — an offline device or a terminal-state task would accumulate forever; a scheduled reap now bounds this. The companion's restart reconciliation loop also resent every receipt it had ever written on every run, forever; it now skips receipts the server already acknowledged. Found during an independent architecture-review pass (Opus, read-only) that also caught and fixed a self-consistency break in the previous commit (two files the tracked device sources import were left untracked) and an invalid CI YAML indentation (the security job had never actually parsed). Local code and automated verification; no production rollout.

#Revision 20 — scheduled retention sweep and bounded restart reconciliation

Retention. Nothing in web/src/lib/device-control.ts ever deleted a row: terminal task states (succeeded/failed/cancelled/expired/unknown) and consumed/expired pairing codes stayed forever, and a device that never reconnects would leave stale awaiting_approval/approved/running tasks past their 5-minute expiry indefinitely — poll only sweeps the polling device's own queue on connect (device-control.ts:121). DeviceControl.reap(retentionDays) is a new maintenance method (not request-scoped, not account-scoped): it first runs the same expiry sweep as poll but across every device regardless of who connects, then deletes terminal tasks and consumed/expired pairings older than the cutoff. user_devices rows are never touched — revoked_at is the audit trail, not disposable. Wired into the existing daily tick (web/src/app/api/admin/tick/route.ts), same shape as the analytics-retention block already there (env var default, try/catch, log only on nonzero change); runs unconditionally, since the tables always exist (revision 18) and reaping is a harmless no-op when the pilot has never been used. Default: DEVICE_TASK_RETENTION_DAYS (30).

Bounded reconciliation. The companion's restart loop (src/devices/agent.ts) read every <taskId>.json receipt in %LOCALAPPDATA%\KIWI\companion\receipts and re-POSTed its result to agent/result on every run, forever, regardless of whether the server had already accepted it — errors were swallowed (.catch(() => {})), so this was invisible growth. local-executor.ts gains markAcknowledged(stateDir, taskId), writing an acknowledged: true flag onto the existing receipt (the file itself, and its path/afterSha256/result, are untouched — restore is independent of whether the server was already notified, so listRestorable still returns acknowledged entries). agent.ts calls it after a successful agent/result in both the restart-reconciliation loop and the normal per-task send, and the reconciliation loop now skips any receipt already marked acknowledged.

New tests: web 4 in web/tests/device-retention.test.ts (old terminal tasks deleted, live and recent-terminal tasks survive, across two accounts; an approved-but-never-polled task is swept to expired and only deleted once past retention; old consumed/expired pairings deleted, a fresh one survives; user_devices never touched), engine 1 in tests/device-execution.test.ts (markAcknowledged preserves the receipt's other fields and restore still works on an acknowledged entry). Verification: 138 engine tests (137 passed + one optional semantic skip; +1); 237 mind/web tests in 37 suites (+4); both typechecks passed. No production rollout.

Revision 19 (2026-09-09): abuse limits now cover every mutating device route (previously only pairing preview/claim), a per-device pending-task ceiling bounds an unattended queue, and the companion CLI gained a restore mode that reverts a previously executed edit from its local backup. Local code and automated verification; no production rollout.

#Revision 19 — rate limits on every mutating route, a pending-task ceiling, and backup restore

Abuse limits. Only pairings/preview/pairings/claim were rate-limited (20 req/min/IP). Every other mutating route was open: a signed-in account could call /tasks, /tasks/decision, /revoke without limit, and an authenticated device could hammer /agent/poll|verify|result|revoke. web/src/app/api/devices/[[...path]]/route.ts adds a limited(key, limit) helper (60s window, reusing the existing rateLimit) on every remaining mutation: browser routes keyed by account id — pairings 10/min, tasks 30/min, tasks/decision 60/min, revoke 20/min; agent routes keyed by device id, not IP (every agent polls through the same tunnel origin) — agent/poll 120/min, agent/verify and agent/result 30/min, agent/revoke 20/min.

Pending-task ceiling. DeviceControl.propose in web/src/lib/device-control.ts now refuses a new proposal (429) once a device already has MAX_PENDING_TASKS_PER_DEVICE (20) tasks in awaiting_approval with an unexpired deadline — the per-minute rate limit alone did not bound how large an unattended queue could grow (each task can carry up to 64 KiB), and this is independent of and reached before the per-minute tasks limit in normal use.

Backup restore. The companion previously wrote a .backup file before every existing-file edit (src/devices/local-executor.ts) but had no way to read it back — "No automatic restore operation exists yet" (revision 6). The execution receipt (<taskId>.json) now also records the edited path and the written content's afterSha256, alongside the existing digest/executionId; both writes (the initial wx receipt and the final result update) carry them, and existing receipts without them are treated as pre-restore-support and refused explicitly rather than guessed at. listRestorable(stateDir) scans receipts for a recorded path, an outcome of succeeded or unknown (only these reach the write step), and a matching .backup file — a brand-new file (no prior content) is correctly excluded, it has nothing to restore. restoreFromBackup(taskId, {workspace, stateDir, confirm}) re-resolves the target through the same safeTarget traversal/symlink guard as execution, refuses if the current file no longer matches the afterSha256 this exact execution wrote (someone changed it since — restoring would silently discard that newer change), shows a before/after preview through the same typed-confirmation pattern as APPLY/UNPAIR, and writes atomically (temp file + rename). The backup file is kept after a restore, so the same recovery point can be reviewed or reused again. src/devices/agent.ts exposes this as a new restore CLI mode (lists restorable edits, asks which one, previews, requires typing RESTORE); --help and the mode prompt mention it.

New tests: engine 5 in tests/device-execution.test.ts (restore reverts content and keeps the backup; declines without writing when not confirmed; refuses when the file changed since the edit; listRestorable excludes new-file edits and pre-restore-support receipts; excludes cancelled/no-receipt tasks), web 3 in web/tests/device-rate-limits.test.ts (pairing creation limited per account, not shared across accounts; the pending-task ceiling triggers before and independent of the per-minute proposal limit; agent endpoints are limited per device). Verification: 137 engine tests (136 passed + one optional semantic skip; +5); 233 mind/web tests in 36 suites (+3); both typechecks passed. No production rollout.

Revision 18 (2026-09-09): GDPR account deletion now clears device-pilot rows and the pilot's tables gained Drizzle/PG schema parity; fixing this surfaced and corrected an unrelated pre-existing defect that made the whole GDPR delete endpoint fail for every account. Local code and automated verification; no production rollout.

#Revision 18 — device rows in account deletion, PG parity, and a GDPR transaction defect

web/src/lib/db/{sqlite-schema.ts,pg-schema.ts} now declare devicePairings/userDevices/deviceTasks, matching device-control.ts's raw SQL exactly (columns, unique constraints, the user_devices/device_tasks foreign keys, all three indexes). web/src/lib/db/index.ts creates the three tables unconditionally at startup, the same way message_files and the other shared tables are created — previously only DeviceControl's own constructor created them, lazily, the first time the (disabled-by-default) devices API was actually invoked; an account that deleted itself before ever opening Dashboard → Devices would have hit a missing-table error on that unrelated code path. web/src/app/api/gdpr/delete/route.ts now deletes device_tasks first (it references user_devices without ON DELETE CASCADE, and foreign_keys=ON), then user_devices, then device_pairings, inside the existing account-deletion transaction.

Writing a test for that surfaced a confirmed, pre-existing defect unrelated to devices: the transaction used db.transaction(async (tx) => { await tx.delete(...) }). better-sqlite3's synchronous transaction wrapper rejects an async callback outright (Transaction function cannot return a promise) — every GDPR deletion request for every account, across all eleven original tables, was failing with HTTP 500. No test had ever exercised this route before. Fixed by making the callback synchronous and calling the query builder's sync .run() on each delete instead of awaiting it.

New tests: web/tests/gdpr-delete-devices.test.ts (2 — cross-account isolation of device rows on deletion; deletion succeeds for an account that never used the devices API, the scenario the lazy-creation bug would have broken). Verification: 132 engine tests unchanged; 230 mind/web tests in 35 suites (+2); both typechecks passed. No production rollout.

Plan revision 6, 2026-09-08. Implemented locally behind KIWI_DEVICES_ENABLED=1; default off. Not deployed, not accepted on two PCs. P0 remains open. This isolated pilot is acceptance-test infrastructure, not permission to skip release gates.

#Scope and installation

Dashboard → Devices creates a single-use pairing code valid for ten minutes. A Windows Node.js 20+ companion verifies the account and asks the local user to accept an existing workspace. It polls outward over HTTPS (HTTP allowed only for loopback development), using an independent revocable device token. Pairing codes and tokens are stored only as SHA-256 hashes on the server; the local credential is encrypted with Windows CurrentUser DPAPI.

Build from the repository root with npx tsc -p tsconfig.companion.json; pack with npm pack ./packages/companion; install the resulting archive with npm install -g <archive.tgz>. Launch kiwi-device pair, then kiwi-device run. With no argument the terminal offers pair/run/status/unpair. --help and --version do not require TTY. No npm lifecycle launches an application. The package is private, version 0.1.0, named @kiwi-local/companion; registry ownership/publication is not established. It runs compiled JavaScript with no runtime dependency on the development repository.

The pilot supports one UTF-8 file create/replace per task, at most 64 KiB, inside the paired workspace. Existing files require their expected lowercase SHA-256; new files use null. Inspect an existing hash with PowerShell Get-FileHash -Algorithm SHA256 -LiteralPath <file>. Commands, deletion, programs, arbitrary disk access and a full coding TUI are not implemented.

#Approval and execution

The owner explicitly selects a device and reviews path/content on the website. Approval binds owner, device, canonical action digest and a five-minute task expiry. The agent displays before/after content and requires APPLY, then checks server authorization and the current local hash again. Refusal, changed contents, expired authorization and revocation prevent a new write.

Before execution the agent records a receipt; retries never repeat the operation. Existing files get a backup. New files use an atomic no-replace operation. After a disconnect/restart the agent reports the saved result or unknown; the server never redelivers a running task. Revocation cancels queued tasks and marks running ones unknown. It cannot undo an already-started write. A manual, operator-invoked restore mode (revision 19) can revert a succeeded/unknown edit from its backup, refusing if the file changed again since; there is still no automatic/server-initiated restore.

Relative traversal, absolute paths, Windows reserved names, observed symlinks/junctions and hard-linked targets are rejected. Existing parent directories are required. Checks are not certified against a hostile concurrent local process replacing paths between checks. Profile receipts/backups are under %LOCALAPPDATA%\KIWI\companion\receipts. kiwi-device unpair revokes the token and removes the encrypted local credential after UNPAIR, retaining backups. Uninstall with npm uninstall -g @kiwi-local/companion.

#API and SQLite

Base: /api/devices; JSON bounded to 100,000 bytes during streaming reads. Disabled routes return 404. Browser mutations require a session, no Authorization header and the site Origin configured by AUTH_URL/NEXTAUTH_URL (request origin only as a development fallback); forwarded Host cannot override it. Exact POST pairing/agent endpoints defer authentication from middleware to the route.

EndpointAuthorizationPurpose
GET /api/devicesAccount sessionOwn devices and last 100 tasks
POST /pairingsAccount session + Origin, 10 req/min/accountIssue one-time code
POST /pairings/preview, /pairings/claimPairing code, 20 requests/min/IPPreview account; atomic Windows pairing
POST /tasksAccount session + Origin, 30 req/min/account, 20 pending/device{deviceId, action:{kind:'write_file',path,content,expectedSha256}}
POST /tasks/decisionAccount session + Origin, 60 req/min/account{id,digest,approve:boolean}
POST /revokeAccount session + Origin, 20 req/min/account{deviceId}
POST /agent/pollDevice Bearer token, 120 req/min/deviceClaim one approved task, at most once
POST /agent/verifyDevice Bearer token, 30 req/min/deviceRecheck {id,executionId,digest} immediately before writing
POST /agent/resultDevice Bearer token, 30 req/min/deviceSame identity plus {outcome,message}; identical receipts are idempotent
POST /agent/revokeDevice Bearer token, 20 req/min/deviceRevoke this device only

web/src/lib/device-control.ts creates device_pairings, user_devices, device_tasks in its constructor; web/src/lib/db/index.ts also creates them unconditionally at web startup (revision 18), so they exist even before the disabled-by-default API is ever used. Text IDs, account foreign keys, hashed credentials, owner/queue indexes; task timestamps are integer milliseconds. States: awaiting_approval, approved, running, succeeded, failed, cancelled, expired, unknown. SQLite transactions serialize pairing, decisions, claiming and result writes. A device is capped at 20 unexpired awaiting_approval tasks (revision 19). DeviceControl.reap (revision 20), run from the daily tick, deletes terminal-state tasks and consumed/expired pairings past DEVICE_TASK_RETENTION_DAYS (default 30) — user_devices rows are kept regardless, as the revocation audit trail. Drizzle/PG schema parity exists as of revision 18 (devicePairings/userDevices/deviceTasks); GDPR account deletion clears all three by user ID. Contents/results are private account data, not shared-room input.

#Evidence and remaining gates

  • Real temporary SQLite tests cover two owners/devices, code expiry/replay, exact decisions, restart, no redelivery, result idempotency and revocation. Handler integration performs a real temporary Windows file edit for A while B cannot see/execute it; auth and local approval are simulated.
  • Middleware tests ensure companion authentication reaches the route while browser approval paths still require login. Executor tests cover denial, tampering, changed files, backups and a real Windows junction. DPAPI round-trips a synthetic credential.
  • Current web validation: 177 mind/web tests in 27 suites, web typecheck and companion compilation passed. Prior engine baseline: 80 passed, one optional semantic test skipped, engine typecheck passed. Earlier companion archive installation/help/version/uninstall passed in an isolated temporary prefix; later internal fixes were recompiled/tested, without repeating archive installation. No live credentials or real user files were used.
  • Revision 6: isolated Next production build and real Edge/HTTP probe passed. Two synthetic signed account sessions paired distinct logical devices; the UI approved A's exact edit, B could not approve/view it, a real temporary file changed only in A's workspace, results survived probe-server restart, denied work was not executed and revocation returned 401. Local APPLY was simulated on this one PC. See reproducible commands and retained evidence.
  • Remaining: two actual Windows systems, interactive/network interruption tests, clean-machine install/update, signed distribution, code review and controlled production rollout. Revision 19 closes the open request/queue abuse-limit gap and adds a manual backup-restore path (both in-memory/single-process only — a multi-instance production deploy still needs a shared rate-limit store, per web/src/lib/rate-limit.ts's own comment); revision 20 closes the row-growth and unbounded-resend gaps in data lifecycle. Pairing UI currently requires the locally built package; no public download exists. P2–P6 acceptance remains open.

Sources: src/devices/, packages/companion/, web/src/lib/device-control.ts, web/src/app/api/devices/[[...path]]/route.ts, web/src/app/(dashboard)/dashboard/devices/, state contract.

~ End of Document ~