KIWI
Autonomous local AI agent and community of voices living in The Grove with private memory.
The Problem
Autonomous AI agents often suffer from leaky memory isolation, unpredictable side-effects on host systems, and lack of human-in-the-loop governance for high-stakes file operations and external device actions.
My Role
Architected and implemented the cognitive engine ("The Grove"), fail-closed host action safety policies, immutable exact-action plan ledgers, multi-device agent companion protocol, and unified multi-channel gateways.
Highlights
- Community of six voices living in The Grove with local memory running on the user's machine
- Local cognitive core ("The Grove") separating owner memory from credentialed client sessions
- Fail-closed security policy requiring immutable exact plans with rollback for all destructive actions
- DPAPI-secured Windows companion agent with local execution receipts and anti-re-execution guarantees
- Multi-modal interaction surface: Next.js Web UI, Voice HUD (:4321), and Telegram bot gateway with inline approvals
- Deterministic audit ledger tracking every state mutation, file movement, and memory association
Stack
Constraints
- Host safety is non-negotiable: no autonomous file deletion or batch reorganization without human-confirmed exact plans.
- Offline-first engine: core cognitive loops and memory databases must run purely locally without cloud dependencies.
- Multi-tenant memory isolation: administrative sessions and API token holders must never access the owner private mind context.
System Architecture
Key Trade-offs
The decisions worth defending — what I chose, what I turned down, and why.
File management execution model
Chose
Immutable two-phase exact-action plans with hash verification and dry-run preview
Rejected
Direct agent tool execution with broad shell permissions
Agents making speculative shell or file modifications risk catastrophic data loss; two-phase plans guarantee human oversight and rollback capability.
Engine-to-web communication architecture
Chose
Dedicated loopback HTTP/SSE gateway with bearer token authentication
Rejected
Monolithic single-process Next.js application
Decoupling the AI engine process from the Next.js frontend ensures background monitoring, telegram polling, and long-running skills survive web redeployments.
Device pairing security model
Chose
Windows DPAPI machine-bound tokens with polling architecture
Rejected
Inbound SSH or remote socket servers on client machines
Inbound listening ports expose clients to local network scans; outbound HTTPS polling behind DPAPI keeps remote companion devices secure.
What I'd Do Differently
An honest retrospective — the stuff I'd change with more time, more users, or a second pass.
- 1Implement automated regression tests for memory graph decay sweeps earlier in the development lifecycle.
- 2Consolidate SQLite schema migrations between better-sqlite3 and pg-schema under a single unified migration runner.
- 3Add fine-grained telemetry dashboards to observe cognitive latency across multi-turn reasoning chains.
Interactive Autonomous Agent Shell
Test live KIWI autonomous agent supervisor routines (agent status, grove ping, mcp tools, help) in the emulated shell below.
Technical Deep-Dive
Architecture, specifications, and implementation details.
P0 — Host action authorization
Revision 17 (2026-09-09): file_organizer closes its exact-plan flow — the mind automation becomes preview-only (never moves files) and the builtin file_organizer skill is the only mover, gated by a validated exact plan + human approval; dev_task becomes fail-closed by default (deny in config). Local code and automated verification; no production restart or deployment.
#Revision 17 — File organizer exact plan and dev_task fail-closed default
file-organizer (the mind automation) has never been allowed to move files (revisions 7/14/16: non-dryRun denied with the exact-plan reason). Revision 17 provides the safe executor it was waiting for:
- Shared
src/skills/fileOrganize.ts:EXTENSION_MAPcategories, plausible destination tree,archiveAfterDays→_archive;computeOrganizeMoves(dir, archiveAfterDays)returns{ moves:[{from,to,size,archived}], skipped }with collision-safe(2)targets checked against both the claimed set and the disk; unclassified files/directories are counted and left in place. - Builtin
file_organizerskill (kindfile_organizer, registered insrc/skills/builtin/index.ts): withdryRun:trueit stages an exact plan (recovery copies included) viaprepareExactPlanand returns the samedata:{ plan, exactPlan:{id,kind}, exactPlanId, backupDir }shape as revision 16; execution requiresexactPlan:{id,kind:'file_organizer'}validated throughvalidateExactPlanand runsexecuteExactPlan; without a reference the skill returns the exact-plan block reason. - Mind automation
file-organizerv1.1.0 is preview-only: it renderscomputeOrganizeMovesoutput for the operator and always points to the chat-approved exact plan; it never touches the disk. Policy:file_organizerbuiltin →allowwhendryRun:true, elseconfirm(with a valid ref) — same deny-without-ref shape as the other exact-plan skills; the automation skill keeps its deny-on-non-dryRun path. dev_taskfail-closed default:src/config/config.tsnow shipssecurity.skillPolicy.dev_task:'deny'(wasconfirm);dev_status/dev_cancelrefuse whenever dev mode is off;DevManager.starthas no active production caller, so no background coding job can start. The safe replacement stays the P1 device-task / P2 interactive-coding flow — a move/delete ledger is not a fit for interactive code edits.- Registry-level block extended:
prepareExactPlanstage refuses an unknownfile_organizerkind like the other kinds.
Evidence: 4 new engine tests in tests/exact-plan.test.ts (file_organizer dryRun prepares + shows targets; non-dryRun without a plan is denied; execution with a real validated plan + approval runs the moves; on-disk revalidation still applies) and 2 new in tests/dev.test.ts (default config denies dev_task; dev_status/dev_cancel fail-closed without dev mode). Engine suite 132 tests (131 passed + 1 optional semantic skip); mind/web 228 in 34 suites (see audit); both typechecks passed.
#Revision 16 fix — exact-action plans for documents_organize and disk_doctor reap
documents_organize (non-dryRun) and disk_doctor reap were blocked (revisions 7/10) with the "approved exact plan" reason. Revision 16 replaces the block with a persistent exact-plan mechanism that keeps mandatory human approval at execution.
Mechanism (src/exactions/ledger.ts, new):
- Immutable plan:
{ id: 'ep-<timestamp36>-<hex>', kind: 'documents_organize' | 'disk_doctor_reap', ops }, operations carry absolute paths plussizeandsha256(only for moves, files ≤16 MB) — verifiable immutability. prepareExactPlan: validates every op (source exists, target free,size>0), takes recovery copies for moves (data/exactions/<id>/moves/N.bin, underKELVIN_DATA_DIR) and writes the ledgerdata/exactions/ledger.jsonwithstatus:'prepared'.validateExactPlan: before any execution re-checks immutability (canonical sha256 over{kind, ops}), backup presence and on-disk state — the source must still exist, the destination must still be free.executeExactPlan: move =rename; delete =renameintobackupDir/deletes/N(norm, instantly recoverable). First error →status:'partial'+{ok:false, done, error}report; success →status:'done'withexecutedAt.rollbackExactPlan: restores from backup (idempotent);rolledbackplans cannot be re-loaded.describeExactPlanrenders the operator preview.
Policy (src/security/host-action-policy.ts):
exactPlanRefOf(args)accepts onlyexactPlan:{id, kind}.legacyActionBlockReason: non-dryRundocuments_organizeanddisk_doctor reaprefuse ONLY without a valid reference; the builtinfile_organizernon-dryRun without aexactPlanref and the automationfile-organizernon-dryRun refuse with the same reason;dev_taskremains unconditionally refused (configdeny).minimumPolicy: preparing staysallow—documents_organizewithdryRun:trueanddisk_doctorwithaction:plan_reap/report; execution with a validexactPlan→confirm(mandatory human approval viaConversationManager, never auto-execution). Without a ref → deny.
Skills:
documents.ts:documents_organizewithdryRun:trueprepares the plan (LLM classification, collision-free " (2)"/" (3)" target names, recovery copies) and returnsdata:{ plan, exactPlan:{id,kind}, exactPlanId, backupDir }; non-dryRun requiresexactPlan, validates against the ledger and executes. The brain check only runs in the prepare path (execution needs no LLM).diskDoctor.ts:action:'plan_reap'stages the delete plan (no deletion);action:'reap'requiresexactPlan, validates and moves folders into backup. The old destructivermreap was removed.
Evidence: 11 new tests in tests/exact-plan.test.ts — empty/missing-source plans refused; move prepare→validate→execute→rollback; destination-collision invalidation; reap-style delete with backup+rollback; partial failure marks partial; full documents_organize flow (dryRun prepares, execution only with ref); disk_doctor plan_reap/reap flow; deny→confirm policy with/without ref; real approval through ConversationManager; exactPlanRefOf shape validation; registry-level block without a plan. Engine suite 124 tests (123 passed + 1 optional semantic skip; +11); mind/web 222 in 33 suites (unchanged); both typechecks passed.
Revision 15 (2026-09-09): plugin skills declare their capabilities; undeclared ones are blocked. Local code and automated verification; no production restart or deployment.
#Revision 15 fix — declared capabilities for plugins (fail-closed)
Revision 14 left plugins without effect metadata: hostSkillPolicy did not know them and fell back to an implicit allow (plugins are not in minimumPolicy), while direct registry.run callers (document skills in src/grove-api.ts:941, CLI diagnostics in src/cli/kiwi.ts:302) bypassed the full policy — an arbitrary plugin from pages/plugins could run with no host-effect checks at all.
Corrections:
- Declarative
Skill.capabilities(read/write/exec/network/system/media,src/skills/types.ts);plugins/example-dice.mjsdeclares['read']. hostSkillPolicytakes a sixthdeclared?: DeclaredSkillInfoparameter (src/security/host-action-policy.ts);declaredSkillPolicy: none/emptycapabilities→confirm, read-only →allow, any mutational effect →confirm. A stale configuredallowcannot weaken the fail-closed path (the revision-7 compositiondeny>confirm> allow is kept; plugins are decided after it).SkillRegistry.skillDeclaration(src/skills/registry.ts) exposes the metadata into every policy path:Agent.declaredSkill(JSON/stream/offline,src/agent/agent.ts),ConversationManager(src/conversation/manager.ts), direct Telegram (src/telegram/run.ts).- Backstop in
registry.run: a plugin registered with asourceFilebut nocapabilitiesis refused beforerun()(ok:false, capabilities reason), so direct callers cannot dodge policy. Declared plugins remain runnable throughagent.runSkill/approval (same as confirmed execution); built-ins and skills without asourceFileare not plugins and are unchanged. src/skills/plugins.tslogs a warning when a plugin loads withoutcapabilities.
Evidence: 8 new tests in tests/plugin-capabilities.test.ts (fail-closed with a stale allow; configured deny; tool scope; skillDeclaration built-in vs plugin; backstop that never reaches run(); direct registry.run on a custom built-in stays allowed). Engine suite 113 tests (112 passed + 1 optional semantic skip; +8); mind/web 222 in 33 suites (unchanged); both typechecks passed.
Revision 14 (2026-09-09): closes the mind-automation path (mind.hear) and the documents root. Local code and automated verification; no production restart or deployment. Revisions 7 and 10 remain as history below.
#Revision 14 fix — policy on mind automation and the documents root
The only production caller of mind.hear is the brain-down fallback in src/grove-api.ts:122 (the scoped route returns 503 while the brain is down). The automation branch in hear() dispatched SkillRunner.execute with ALL permissions (FileRead/FileWrite/Network/System/Database/Notification) and dryRun:false, with no host-action policy — any future caller (the ≤120s brain handshake window, other paths) could launch applications through execSync with injectable arguments and move/write files without approval.
Corrections in src/security/host-action-policy.ts + src/mind/kelvin.ts + src/mind/automation/skills/app-launcher.ts:
minimumPolicynow classifies the automation skills:app-launcherandnote-taker→confirm; non-dryRunfile-organizer→denywith the exact-plan reason (legacyActionBlockReason); read-only diagnostics (web-search,system-monitor,file-finder,daily-briefing,timer,reminder) stayallow.- The automation branch in
mind.hearapplieshostSkillPolicybefore any skill runs:deny→ refusal with the reason;confirm→ refused like an internal thread ("fallback channel", no one can approve) withautomationResult: 'blocked', persisted to memory as blocked. Unknown skills → explicit error. - Least privilege:
ctx.permissionsis now exactlyskillDef.permissions, not the full grant list. app-launcherhardened:isSafeLaunchTokenrejects shell metacharacters ($(){}!&|<>^%``"',;) inappNameandargsbeforeexecSync` — hostile paths/arguments never reach the shell.
Documents root (POST /api/documents/root in src/grove-api.ts) ran mkdirSync(target, {recursive:true}) on any path from the request, then saveConfig plus a live update — it could create folders anywhere and point the live document indexer at them. resolveDocumentsRoot (src/documents/request-scope.ts) now refuses creating new folders outside PROJECT_ROOT (HTTP 400); selecting an existing folder anywhere stays allowed (owner-managed — the current deployment uses C:\Users\alexm\Documents\KiwiDocs, which exists and remains selectable). Invalid/control paths get 400.
#Revision 10 fix — /api/documents/organize broker
Further inspection found engine route POST /api/documents/organize executing real file moves with dryRun=false without passing through the host-action policy (the legacyActionBlockReason refusal for non-dryRun documents_organize only applied inside registry.run). The route now refuses any dryRun !== true with HTTP 403 and the same message as the registry; the web never calls this route. It remains dryRun-only like the skill; real moves stay in the exact approved P1 plan.
#Confirmed failures
Nine new regression tests failed before the changes: the model could execute tools outside its advertised list (including streaming); an empty list did not disable tools; caller allow bypassed configured deny; offline fallback ignored both policies and tool scope; pending confirmation forgot the originating policy; host mutations could use old allow settings; previews cut arguments at 100 characters; registry dispatch still permitted broad bulk actions.
An additional scenario verifies that generated routine steps cannot confirm a preceding mutation. Direct Telegram inspection found unconfirmed /vm start and case variants of PC actions, /docs organize move dispatching directly, and /dev starting OpenCode after only a broad task approval.
#Enforced behavior
src/security/host-action-policy.ts composes restrictions: denial wins, then confirmation, then allow. Caller gates can narrow access, never weaken configured denial or the mandatory host policy. tools is an execution boundary in JSON and streaming agent loops; [] advertises and permits no tools.
Known program, command, file-write, clipboard-write, power, wake, VM/process mutation, backup mutation, schedule mutation and routine create/delete actions require confirmation even with old allow configuration. Read diagnostics remain available subject to configured restrictions. This is an explicit list of known built-ins; it does not certify arbitrary plugins as read-only.
ConversationManager applies the same gate to offline fallback and pending execution. Each pending entry keeps cloned arguments, original channel identity, five-minute expiry and the originating gate; both originating and current policies are rechecked at confirmation. Internal scheduler/routine/auto-resolve/document threads refuse confirmation-requiring actions rather than treating generated text as consent. Pending approvals still disappear on restart.
The generic preview serializes the complete skill and arguments, identifies the operator host, and escapes controls/Markdown backticks and directional controls. A preview exceeding 2,800 characters produces an explicit refusal and creates no approval; it is never truncated for consent. This names requested arguments, not every downstream effect of an executable. Rich file diffs, immutable resolved paths/configuration and before-state hashes remain part of the P1 migration.
Direct Telegram registry calls now use one policy helper. It clones arguments, issues the existing private single-use buttons when needed and rechecks policy before dispatch after approval. Explicit denial also applies to direct diagnostics. No live Telegram messages were sent during testing.
#Broad actions withheld pending exact plans
| Entry | Current result | Replacement |
|---|---|---|
dev_task, Telegram /dev <task> | Explicit refusal; dev.enabled=true cannot start a job | Reviewed edits/commands through shared tasks and the P2 coding flow (in P1) |
documents_organize | Exact plan staged on dryRun:true (recovery copies); execution only with a valid exactPlan + approval (revision 16) | Implemented — no KiwiDocs/deployment folders have run plans yet |
disk_doctor with action:reap | Non-dryRun refused without a ref; plan_reap stages a recoverable plan; reap+exactPlan moves into backup (revision 16) | Implemented — previously deletable items were never deleted |
file-organizer (mind automation) | Preview-only since revision 17: dry-run reports targets and never touches the disk; non-dryRun refused with the exact-plan reason | Builtin file_organizer exact plan approved in chat, backed up before moving (implemented — one move path left: operator approval) |
file_organizer (builtin skill) | dryRun:true stages an exact plan (allow); execution only with a validated exactPlan + approval (confirm), no ref → deny (revision 17) | Implemented — no run plans in production yet |
Registry dispatch blocks these operations before a skill body runs; built-in entrypoints also refuse them. The /api/documents/organize broker route now applies the same refusal (revision 10), and the web does not call it. The low-level DevManager and document library remain as migration code, with no application caller launching DevManager jobs. Synthetic subprocess tests exercise that runner directly. dev_status and confirmed dev_cancel remain available for legacy jobs. Existing configuration files and libraries are preserved.
#Evidence and remaining work
tests/mind-automation-policy.test.ts(new, revision 14): automation classification (app-launcher/note-taker→ confirm;file-organizernon-dryRun → deny, dryRun → allow;web-search/system-monitor/file-finder→ allow), the exact-plan reason forfile-organizer, and metacharacter refusal inapp-launcher(appName and args) without reachingexecSync; dry-run stays permitted.tests/document-scope.test.ts(extended, revision 14):resolveDocumentsRootrefuses a missing path and creation outside the project folder, allows creation inside it and selection of an existing folder anywhere.- Full engine suite (revision 14): 105 tests, 104 passed + one optional semantic skip (up 4 from revision 13). Mind/web: 222 passed in 33 suites (unchanged — no web code). Engine typecheck passed.
- Full engine suite (revision 15): 113 tests, 112 passed + one optional semantic skip (up 8 from revision 14 —
tests/plugin-capabilities.test.ts). Mind/web: 222 passed in 33 suites; both typechecks passed. - Full engine suite (revision 16): 124 tests, 123 passed + one optional semantic skip (up 11 from revision 15 —
tests/exact-plan.test.ts). Mind/web: 222 passed in 33 suites (unchanged); both typechecks passed. tests/exact-plan.test.ts(new, revision 16): full move and delete lifecycle with backup/rollback, collision invalidation, partial failure,documents_organizeanddisk_doctorskill flows, deny→confirm policy and real manager approval.tests/host-action-boundary.test.ts: 10 passing regressions, synthetic counters instead of real host mutation; the initial nine all failed before correction.tests/mind/unit/telegram-owner-channel.test.ts: 8 passing mocked-transport cases, including VM/PC approval, denial changed after proposal and blocked bulk/coding requests.- Real temporary-file tests now require consent before emitting a saved-note artifact; document classification preserves source files and rejects actual movement.
Remaining P0 (revision 17): dev_task stays blocked — config default deny, dev_status/dev_cancel fail-closed, DevManager.start without a production caller; its safe replacement is the P1 device-task / P2 interactive-coding flow, deferred to P1. Two-Windows acceptance is blocked (no second Windows; VMs are Linux). documents_organize, disk_doctor reap and file_organizer are now allowed only with an exact plan + approval and have run no real moves/deletions in production. The engine remains operator-owned; it is never a fallback for website users' devices. Direct low-level skill calls by trusted application code are not a public authorization API.
Sources: src/exactions/ledger.ts (new), src/skills/fileOrganize.ts (new), src/skills/builtin/fileOrganizer.ts (new), src/agent/agent.ts, src/conversation/manager.ts, src/skills/registry.ts, src/skills/builtin/{documents,diskDoctor}.ts, src/security/host-action-policy.ts, src/mind/kelvin.ts, src/telegram/run.ts, state contract, checkpoint.