Skip to content
←Return to Projects
Case Study

KIWI

Autonomous local AI agent and community of voices living in The Grove with private memory.

The Problem

Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.

My Role

Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.

Highlights

  • Community of six voices living in The Grove with local memory running on the user's machine
  • Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
  • Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
  • DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
  • Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
  • Deterministic audit ledger tracking every state mutation, file movement, and memory association

Stack

TypeScriptNext.js 15Node.jsSQLiteDrizzle ORMDPAPITelegram Bot APIDocker

Constraints

  • Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
  • Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
  • Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.

System Architecture

Interaction Channels
Next.js 15 Web App (:3333)
Voice HUD (:4321)
Telegram Bot
Embed Bridge (:3234)
Gateway & Protocol
Private Grove API (:8431)
Device Control API
DPAPI Token Auth
Cognitive Engine ("The Grove")
Mind Core
Behavior & Persona
Memory Tiering (STM/MTM/LTM)
Skill Registry
Execution & Safety
Host Action Policy
Exact-Action Ledger
Backup & Rollback Ledger
Distributed Node Runtime
Windows Companion Daemon
Local File Executor
SQLite / Drizzle State

Key Trade-offs

The decisions worth defending — what I chose, what I turned down, and why.

File management execution model

Chose

Immutable two-phase exact-action plans with hash verification and dry-run preview

Rejected

Direct agent tool execution with broad shell permissions

Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.

Engine-to-web communication architecture

Chose

Dedicated loopback HTTP/SSE gateway with bearer token authentication

Rejected

Monolithic single-process Next.js application

Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.

Device pairing security model

Chose

Windows DPAPI machine-bound tokens with polling architecture

Rejected

Inbound SSH or remote socket servers on client machines

Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.

What I'd Do Differently

An honest retrospective — the stuff I'd change with more time, more users, or a second pass.

  1. 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
  2. 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
  3. 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.

Interactive Autonomous Agent Shell

Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.

icelegends-interactive-shell ~ [kiwi]
History: ↑ / ↓
kiwi@the-grove:~/voices$

Technical Deep-Dive

Architecture, specifications, and implementation details.

P0 — Host action authorization

Revision 17 (2026-09-09): file_organizer closes its exact-plan flow — the mind automation becomes preview-only (never moves files) and the builtin file_organizer skill is the only mover, gated by a validated exact plan + human approval; dev_task becomes fail-closed by default (deny in config). Local code and automated verification; no production restart or deployment.

#Revision 17 — File organizer exact plan and dev_task fail-closed default

file-organizer (the mind automation) has never been allowed to move files (revisions 7/14/16: non-dryRun denied with the exact-plan reason). Revision 17 provides the safe executor it was waiting for:

  • Shared src/skills/fileOrganize.ts: EXTENSION_MAP categories, plausible destination tree, archiveAfterDays → _archive; computeOrganizeMoves(dir, archiveAfterDays) returns { moves:[{from,to,size,archived}], skipped } with collision-safe (2) targets checked against both the claimed set and the disk; unclassified files/directories are counted and left in place.
  • Builtin file_organizer skill (kind file_organizer, registered in src/skills/builtin/index.ts): with dryRun:true it stages an exact plan (recovery copies included) via prepareExactPlan and returns the same data:{ plan, exactPlan:{id,kind}, exactPlanId, backupDir } shape as revision 16; execution requires exactPlan:{id,kind:'file_organizer'} validated through validateExactPlan and runs executeExactPlan; without a reference the skill returns the exact-plan block reason.
  • Mind automation file-organizer v1.1.0 is preview-only: it renders computeOrganizeMoves output for the operator and always points to the chat-approved exact plan; it never touches the disk. Policy: file_organizer builtin → allow when dryRun:true, else confirm (with a valid ref) — same deny-without-ref shape as the other exact-plan skills; the automation skill keeps its deny-on-non-dryRun path.
  • dev_task fail-closed default: src/config/config.ts now ships security.skillPolicy.dev_task:'deny' (was confirm); dev_status/dev_cancel refuse whenever dev mode is off; DevManager.start has no active production caller, so no background coding job can start. The safe replacement stays the P1 device-task / P2 interactive-coding flow — a move/delete ledger is not a fit for interactive code edits.
  • Registry-level block extended: prepareExactPlan stage refuses an unknown file_organizer kind like the other kinds.

Evidence: 4 new engine tests in tests/exact-plan.test.ts (file_organizer dryRun prepares + shows targets; non-dryRun without a plan is denied; execution with a real validated plan + approval runs the moves; on-disk revalidation still applies) and 2 new in tests/dev.test.ts (default config denies dev_task; dev_status/dev_cancel fail-closed without dev mode). Engine suite 132 tests (131 passed + 1 optional semantic skip); mind/web 228 in 34 suites (see audit); both typechecks passed.

#Revision 16 fix — exact-action plans for documents_organize and disk_doctor reap

documents_organize (non-dryRun) and disk_doctor reap were blocked (revisions 7/10) with the "approved exact plan" reason. Revision 16 replaces the block with a persistent exact-plan mechanism that keeps mandatory human approval at execution.

Mechanism (src/exactions/ledger.ts, new):

  • Immutable plan: { id: 'ep-<timestamp36>-<hex>', kind: 'documents_organize' | 'disk_doctor_reap', ops }, operations carry absolute paths plus size and sha256 (only for moves, files ≤16 MB) — verifiable immutability.
  • prepareExactPlan: validates every op (source exists, target free, size>0), takes recovery copies for moves (data/exactions/<id>/moves/N.bin, under KELVIN_DATA_DIR) and writes the ledger data/exactions/ledger.json with status:'prepared'.
  • validateExactPlan: before any execution re-checks immutability (canonical sha256 over {kind, ops}), backup presence and on-disk state — the source must still exist, the destination must still be free.
  • executeExactPlan: move = rename; delete = rename into backupDir/deletes/N (no rm, instantly recoverable). First error → status:'partial' + {ok:false, done, error} report; success → status:'done' with executedAt.
  • rollbackExactPlan: restores from backup (idempotent); rolledback plans cannot be re-loaded. describeExactPlan renders the operator preview.

Policy (src/security/host-action-policy.ts):

  • exactPlanRefOf(args) accepts only exactPlan:{id, kind}.
  • legacyActionBlockReason: non-dryRun documents_organize and disk_doctor reap refuse ONLY without a valid reference; the builtin file_organizer non-dryRun without a exactPlan ref and the automation file-organizer non-dryRun refuse with the same reason; dev_task remains unconditionally refused (config deny).
  • minimumPolicy: preparing stays allow — documents_organize with dryRun:true and disk_doctor with action:plan_reap/report; execution with a valid exactPlan → confirm (mandatory human approval via ConversationManager, never auto-execution). Without a ref → deny.

Skills:

  • documents.ts: documents_organize with dryRun:true prepares the plan (LLM classification, collision-free " (2)"/" (3)" target names, recovery copies) and returns data:{ plan, exactPlan:{id,kind}, exactPlanId, backupDir }; non-dryRun requires exactPlan, validates against the ledger and executes. The brain check only runs in the prepare path (execution needs no LLM).
  • diskDoctor.ts: action:'plan_reap' stages the delete plan (no deletion); action:'reap' requires exactPlan, validates and moves folders into backup. The old destructive rm reap was removed.

Evidence: 11 new tests in tests/exact-plan.test.ts — empty/missing-source plans refused; move prepare→validate→execute→rollback; destination-collision invalidation; reap-style delete with backup+rollback; partial failure marks partial; full documents_organize flow (dryRun prepares, execution only with ref); disk_doctor plan_reap/reap flow; deny→confirm policy with/without ref; real approval through ConversationManager; exactPlanRefOf shape validation; registry-level block without a plan. Engine suite 124 tests (123 passed + 1 optional semantic skip; +11); mind/web 222 in 33 suites (unchanged); both typechecks passed.

Revision 15 (2026-09-09): plugin skills declare their capabilities; undeclared ones are blocked. Local code and automated verification; no production restart or deployment.

#Revision 15 fix — declared capabilities for plugins (fail-closed)

Revision 14 left plugins without effect metadata: hostSkillPolicy did not know them and fell back to an implicit allow (plugins are not in minimumPolicy), while direct registry.run callers (document skills in src/grove-api.ts:941, CLI diagnostics in src/cli/kiwi.ts:302) bypassed the full policy — an arbitrary plugin from pages/plugins could run with no host-effect checks at all.

Corrections:

  • Declarative Skill.capabilities (read/write/exec/network/system/media, src/skills/types.ts); plugins/example-dice.mjs declares ['read'].
  • hostSkillPolicy takes a sixth declared?: DeclaredSkillInfo parameter (src/security/host-action-policy.ts); declaredSkillPolicy: none/empty capabilities → confirm, read-only → allow, any mutational effect → confirm. A stale configured allow cannot weaken the fail-closed path (the revision-7 composition deny > confirm > allow is kept; plugins are decided after it).
  • SkillRegistry.skillDeclaration (src/skills/registry.ts) exposes the metadata into every policy path: Agent.declaredSkill (JSON/stream/offline, src/agent/agent.ts), ConversationManager (src/conversation/manager.ts), direct Telegram (src/telegram/run.ts).
  • Backstop in registry.run: a plugin registered with a sourceFile but no capabilities is refused before run() (ok:false, capabilities reason), so direct callers cannot dodge policy. Declared plugins remain runnable through agent.runSkill/approval (same as confirmed execution); built-ins and skills without a sourceFile are not plugins and are unchanged.
  • src/skills/plugins.ts logs a warning when a plugin loads without capabilities.

Evidence: 8 new tests in tests/plugin-capabilities.test.ts (fail-closed with a stale allow; configured deny; tool scope; skillDeclaration built-in vs plugin; backstop that never reaches run(); direct registry.run on a custom built-in stays allowed). Engine suite 113 tests (112 passed + 1 optional semantic skip; +8); mind/web 222 in 33 suites (unchanged); both typechecks passed.

Revision 14 (2026-09-09): closes the mind-automation path (mind.hear) and the documents root. Local code and automated verification; no production restart or deployment. Revisions 7 and 10 remain as history below.

#Revision 14 fix — policy on mind automation and the documents root

The only production caller of mind.hear is the brain-down fallback in src/grove-api.ts:122 (the scoped route returns 503 while the brain is down). The automation branch in hear() dispatched SkillRunner.execute with ALL permissions (FileRead/FileWrite/Network/System/Database/Notification) and dryRun:false, with no host-action policy — any future caller (the ≤120s brain handshake window, other paths) could launch applications through execSync with injectable arguments and move/write files without approval.

Corrections in src/security/host-action-policy.ts + src/mind/kelvin.ts + src/mind/automation/skills/app-launcher.ts:

  • minimumPolicy now classifies the automation skills: app-launcher and note-taker → confirm; non-dryRun file-organizer → deny with the exact-plan reason (legacyActionBlockReason); read-only diagnostics (web-search, system-monitor, file-finder, daily-briefing, timer, reminder) stay allow.
  • The automation branch in mind.hear applies hostSkillPolicy before any skill runs: deny → refusal with the reason; confirm → refused like an internal thread ("fallback channel", no one can approve) with automationResult: 'blocked', persisted to memory as blocked. Unknown skills → explicit error.
  • Least privilege: ctx.permissions is now exactly skillDef.permissions, not the full grant list.
  • app-launcher hardened: isSafeLaunchToken rejects shell metacharacters ($(){}!&|<>^%``"',;) in appNameandargsbeforeexecSync` — hostile paths/arguments never reach the shell.

Documents root (POST /api/documents/root in src/grove-api.ts) ran mkdirSync(target, {recursive:true}) on any path from the request, then saveConfig plus a live update — it could create folders anywhere and point the live document indexer at them. resolveDocumentsRoot (src/documents/request-scope.ts) now refuses creating new folders outside PROJECT_ROOT (HTTP 400); selecting an existing folder anywhere stays allowed (owner-managed — the current deployment uses C:\Users\alexm\Documents\KiwiDocs, which exists and remains selectable). Invalid/control paths get 400.

#Revision 10 fix — /api/documents/organize broker

Further inspection found engine route POST /api/documents/organize executing real file moves with dryRun=false without passing through the host-action policy (the legacyActionBlockReason refusal for non-dryRun documents_organize only applied inside registry.run). The route now refuses any dryRun !== true with HTTP 403 and the same message as the registry; the web never calls this route. It remains dryRun-only like the skill; real moves stay in the exact approved P1 plan.

#Confirmed failures

Nine new regression tests failed before the changes: the model could execute tools outside its advertised list (including streaming); an empty list did not disable tools; caller allow bypassed configured deny; offline fallback ignored both policies and tool scope; pending confirmation forgot the originating policy; host mutations could use old allow settings; previews cut arguments at 100 characters; registry dispatch still permitted broad bulk actions.

An additional scenario verifies that generated routine steps cannot confirm a preceding mutation. Direct Telegram inspection found unconfirmed /vm start and case variants of PC actions, /docs organize move dispatching directly, and /dev starting OpenCode after only a broad task approval.

#Enforced behavior

src/security/host-action-policy.ts composes restrictions: denial wins, then confirmation, then allow. Caller gates can narrow access, never weaken configured denial or the mandatory host policy. tools is an execution boundary in JSON and streaming agent loops; [] advertises and permits no tools.

Known program, command, file-write, clipboard-write, power, wake, VM/process mutation, backup mutation, schedule mutation and routine create/delete actions require confirmation even with old allow configuration. Read diagnostics remain available subject to configured restrictions. This is an explicit list of known built-ins; it does not certify arbitrary plugins as read-only.

ConversationManager applies the same gate to offline fallback and pending execution. Each pending entry keeps cloned arguments, original channel identity, five-minute expiry and the originating gate; both originating and current policies are rechecked at confirmation. Internal scheduler/routine/auto-resolve/document threads refuse confirmation-requiring actions rather than treating generated text as consent. Pending approvals still disappear on restart.

The generic preview serializes the complete skill and arguments, identifies the operator host, and escapes controls/Markdown backticks and directional controls. A preview exceeding 2,800 characters produces an explicit refusal and creates no approval; it is never truncated for consent. This names requested arguments, not every downstream effect of an executable. Rich file diffs, immutable resolved paths/configuration and before-state hashes remain part of the P1 migration.

Direct Telegram registry calls now use one policy helper. It clones arguments, issues the existing private single-use buttons when needed and rechecks policy before dispatch after approval. Explicit denial also applies to direct diagnostics. No live Telegram messages were sent during testing.

#Broad actions withheld pending exact plans

EntryCurrent resultReplacement
dev_task, Telegram /dev <task>Explicit refusal; dev.enabled=true cannot start a jobReviewed edits/commands through shared tasks and the P2 coding flow (in P1)
documents_organizeExact plan staged on dryRun:true (recovery copies); execution only with a valid exactPlan + approval (revision 16)Implemented — no KiwiDocs/deployment folders have run plans yet
disk_doctor with action:reapNon-dryRun refused without a ref; plan_reap stages a recoverable plan; reap+exactPlan moves into backup (revision 16)Implemented — previously deletable items were never deleted
file-organizer (mind automation)Preview-only since revision 17: dry-run reports targets and never touches the disk; non-dryRun refused with the exact-plan reasonBuiltin file_organizer exact plan approved in chat, backed up before moving (implemented — one move path left: operator approval)
file_organizer (builtin skill)dryRun:true stages an exact plan (allow); execution only with a validated exactPlan + approval (confirm), no ref → deny (revision 17)Implemented — no run plans in production yet

Registry dispatch blocks these operations before a skill body runs; built-in entrypoints also refuse them. The /api/documents/organize broker route now applies the same refusal (revision 10), and the web does not call it. The low-level DevManager and document library remain as migration code, with no application caller launching DevManager jobs. Synthetic subprocess tests exercise that runner directly. dev_status and confirmed dev_cancel remain available for legacy jobs. Existing configuration files and libraries are preserved.

#Evidence and remaining work

  • tests/mind-automation-policy.test.ts (new, revision 14): automation classification (app-launcher/note-taker → confirm; file-organizer non-dryRun → deny, dryRun → allow; web-search/system-monitor/file-finder → allow), the exact-plan reason for file-organizer, and metacharacter refusal in app-launcher (appName and args) without reaching execSync; dry-run stays permitted.
  • tests/document-scope.test.ts (extended, revision 14): resolveDocumentsRoot refuses a missing path and creation outside the project folder, allows creation inside it and selection of an existing folder anywhere.
  • Full engine suite (revision 14): 105 tests, 104 passed + one optional semantic skip (up 4 from revision 13). Mind/web: 222 passed in 33 suites (unchanged — no web code). Engine typecheck passed.
  • Full engine suite (revision 15): 113 tests, 112 passed + one optional semantic skip (up 8 from revision 14 — tests/plugin-capabilities.test.ts). Mind/web: 222 passed in 33 suites; both typechecks passed.
  • Full engine suite (revision 16): 124 tests, 123 passed + one optional semantic skip (up 11 from revision 15 — tests/exact-plan.test.ts). Mind/web: 222 passed in 33 suites (unchanged); both typechecks passed.
  • tests/exact-plan.test.ts (new, revision 16): full move and delete lifecycle with backup/rollback, collision invalidation, partial failure, documents_organize and disk_doctor skill flows, deny→confirm policy and real manager approval.
  • tests/host-action-boundary.test.ts: 10 passing regressions, synthetic counters instead of real host mutation; the initial nine all failed before correction.
  • tests/mind/unit/telegram-owner-channel.test.ts: 8 passing mocked-transport cases, including VM/PC approval, denial changed after proposal and blocked bulk/coding requests.
  • Real temporary-file tests now require consent before emitting a saved-note artifact; document classification preserves source files and rejects actual movement.

Remaining P0 (revision 17): dev_task stays blocked — config default deny, dev_status/dev_cancel fail-closed, DevManager.start without a production caller; its safe replacement is the P1 device-task / P2 interactive-coding flow, deferred to P1. Two-Windows acceptance is blocked (no second Windows; VMs are Linux). documents_organize, disk_doctor reap and file_organizer are now allowed only with an exact plan + approval and have run no real moves/deletions in production. The engine remains operator-owned; it is never a fallback for website users' devices. Direct low-level skill calls by trusted application code are not a public authorization API.

Sources: src/exactions/ledger.ts (new), src/skills/fileOrganize.ts (new), src/skills/builtin/fileOrganizer.ts (new), src/agent/agent.ts, src/conversation/manager.ts, src/skills/registry.ts, src/skills/builtin/{documents,diskDoctor}.ts, src/security/host-action-policy.ts, src/mind/kelvin.ts, src/telegram/run.ts, state contract, checkpoint.

~ End of Document ~