Extend development documentation

.ai/INDEX.md

@@ -3,14 +3,18 @@
 ## Overview
 
 This is the canonical entrypoint for repository AI context.
-Read this file first at the start of each session, then use the knowledge index to choose the relevant context files.
-Repository AI context lives under `.ai/knowledge/`.
+Read this file first at the start of each session, then use the indexes below to choose the relevant files.
+Repository AI context lives under `.ai/knowledge/`, binding code rules under `.ai/rules/`, and recurring maintenance processes under `.ai/workflows/`.
 
 ## Start Here
 
 1. `.ai/INDEX.md` (this file)
 2. `.ai/knowledge/INDEX.md` (context map and repository guardrails)
 
+Before writing or reviewing code, read `.ai/rules/INDEX.md`.
+
 ## Navigation
 
 - Module and target context: `.ai/knowledge/INDEX.md`
+- Code rules for implementation and review: `.ai/rules/INDEX.md`
+- Recurring maintenance processes: `.ai/workflows/INDEX.md`

.ai/knowledge/INDEX.md

@@ -9,8 +9,8 @@ Use this file to choose the next context file to read. Do not load every context
 
 - Main code lives in the root Rust workspace under `crates/`.
 - `plugins/` is intentionally outside the main workspace validation path.
-- The root `justfile` is the developer entrypoint, but do not run `just` recipes unless the user explicitly asks for them.
-- Run build, test, lint, or validation commands only when the user requests validation or when needed to verify a change.
+- The root `justfile` is the developer entrypoint for build, validation, and run commands.
+- `just validate`, aliased as `just pr`, runs formatting, check, clippy, and tests. It is the full gate CI runs on every pull request, so run it before proposing one. Use targeted `cargo -p <crate>` commands while iterating.
 
 ## Primary Context
 

.ai/knowledge/crates/app/AGENTS.md

@@ -7,13 +7,14 @@ It owns the desktop application shell and integrates directly with the core runt
 
 ## Start Here
 
-1. `crates/app/src/host/ui/`
-2. `crates/app/src/session/ui/`
-3. `crates/app/src/host/service/`
-4. `crates/app/src/session/service/`
-5. `crates/app/src/host/ui/state/`
-6. `.ai/knowledge/crates/core/AGENTS.md`
-7. `.ai/knowledge/crates/stypes/AGENTS.md`
+1. `.ai/knowledge/crates/app/architecture.md` — read before any change that crosses the UI/service boundary, adds an extension point, or is a design review.
+2. `crates/app/src/host/ui/`
+3. `crates/app/src/session/ui/`
+4. `crates/app/src/host/service/`
+5. `crates/app/src/session/service/`
+6. `crates/app/src/host/ui/state/`
+7. `.ai/knowledge/crates/core/AGENTS.md`
+8. `.ai/knowledge/crates/stypes/AGENTS.md`
 
 ## If You Need X, Start Here
 
@@ -36,20 +37,19 @@ It owns the desktop application shell and integrates directly with the core runt
 - Command palette and shortcuts: `crates/app/src/host/ui/command_palette/`, `crates/app/src/host/ui/shortcuts/`
 - Built-in app updater/release checks: `crates/app/src/host/service/update/`, `crates/app/src/host/ui/update.rs`, `crates/app/src/host/ui/state/info.rs`, `crates/app/src/host/ui/banners/update.rs`, `crates/app/src/host/ui/menu.rs`, `crates/app/src/host/service/mod.rs`, `crates/app/src/host/command.rs`, and `crates/app/src/host/message.rs`
 
-## Ownership Model
+## State Owners
+
+For the ownership model and the channels between UI and services, see `.ai/knowledge/crates/app/architecture.md`.
 
-- UI state structs are the source of truth for native application/UI state.
-- Services do not own canonical app/UI state; they only run backend work and copied request data.
 - `Host` is the `eframe::App` and owns top-level rendering, message polling, tabs, storage, notifications, and global UI actions.
 - `HostState` owns active tabs, open sessions, setup tabs, preferences, shortcuts, app info, top-level modals, plugin UI state, and `HostRegistry`.
 - `HostRegistry` owns global filter/search-value definitions and presets.
 - `HostStorage` owns UI-facing service-backed storage domains: recent sessions, file explorer/favorite folders, and app settings. Presets stay in `HostRegistry`, which hands out their save snapshot while `HostStorage` collects one.
 - `persist` owns only lightweight host UI preferences saved through `eframe` storage.
-- `PluginService` owns `PluginsManager`; UI code consumes published `PluginsState` and must not load or mutate plugin runtime state directly.
+- `PluginService` owns `PluginsManager`; the UI consumes published `PluginsState`.
 - `Session` owns one live session UI shell and its component UIs.
 - `SessionShared` owns canonical per-session state: applied filters, temp search, search/chart operation state, log selection, bookmarks, observe state, attachments, exports, signals, and view state.
 - Component UIs should keep only local render/edit/cache state unless they already own a specific domain.
-- Backend/UI integration crosses `HostCommand`/`HostMessage` and `SessionCommand`/`SessionMessage`.
 
 ## Coding Guidelines
 
@@ -73,7 +73,6 @@ It owns the desktop application shell and integrates directly with the core runt
 
 ## Validation
 
-Run validation only when requested or needed to verify a change.
 Use targeted Cargo commands from the repository root, for example:
 
 - Check: `cargo check -p app`

.ai/knowledge/crates/app/architecture.md

@@ -0,0 +1,101 @@
+# Native GUI Architecture
+
+## Overview
+
+`crates/app` runs as a single process: the UI thread runs one `eframe::App`, and all async work runs on one tokio runtime on one spawned thread. There is no separate backend process; the core crates are linked directly.
+
+Read this file before adding a feature, crossing the UI/service boundary, or reviewing a design.
+Binding code rules live in `.ai/rules/app-ui.md`.
+
+## Frame Loop
+
+The workspace patches `eframe`/`egui` to a fork that splits `App::logic` from `App::ui`:
+
+- `logic` drains the host message and notification channels, polls the pending storage save, then pumps the messages of **every** session tab.
+- `ui` renders **only** the active tab.
+
+Background sessions stay alive in state but are never drawn. Message handling belongs in `logic` or `Session::handle_messages`; putting it in render code starves every inactive session.
+
+## Host and Sessions
+
+`Host` is the `eframe::App` and owns tabs, top-level state, storage, notifications, and global actions. `HostTabs` owns the tab vector: Home is index 0 and cannot be closed, and the session count is unbounded.
+
+A session is a pair joined by its own channel pair: `Session` (UI) and `SessionService`, which wraps one `session_core::Session`.
+
+- **Create:** `HostCommand::StartSession` → `HostService::start_session` → `SessionService::spawn` → `HostMessage::SessionCreated` → `Host` builds the `Session` and `HostTabs` adds it, replacing the originating setup tab in place when there is one.
+- **Tab close:** registry cleanup, remove the tab, fire-and-forget `CloseSession`.
+- **App exit:** close every receiver, send `CloseSession` with an ack channel, wait on all acks under one shared deadline, flush storage, then `HostCommand::OnShutdown`.
+
+## Communication
+
+Four async channels:
+
+- `HostCommand` — UI to host service.
+- `HostMessage` — host service to UI.
+- `AppNotification` — service to UI, shown as banners and kept in notification history.
+- `HostAsyncEvent` — internal to the service layer: results of background work the host command loop itself must act on. Never reaches the UI.
+
+Per session, `SessionCommand` and `SessionMessage` mirror the first two.
+
+Services never send on a raw sender. Every UI-bound send goes through `ServiceSenders`, which requests a repaint on success; a raw send arrives but may not be drawn until an unrelated event wakes the UI.
+
+The one exception to async messaging is a `std::sync::mpsc::Sender` embedded in a command, used where the caller must block for the answer: the blocking line grabs and the shutdown acknowledgement.
+
+## Service Layer
+
+`HostService::run` is the only host command loop, selecting over incoming commands and `HostAsyncEvent`. Startup is a blocking handshake: `HostService::spawn` returns `HostServiceInit` over a sync channel, so the first frame already has recent sessions, app settings, and presets. Heavier domains load lazily afterwards.
+
+Sub-services come in two shapes. Work whose result the host loop must act on is a struct initialized with the `HostAsyncEvent` sender (`StorageService`, `PluginService`). Work that only updates the UI is free functions taking `ServiceSenders` by value (`update`). `session/service/export.rs` and `tracker.rs` are not services; they are `impl SessionService` files.
+
+A service may own backend runtime state — `PluginsManager`, `session_core::Session`, the instance temp directory, the operation tracker — but never state the UI renders from. It publishes snapshots instead.
+
+## One Round Trip
+
+Applying a search filter, end to end:
+
+1. The search bar calls `SessionShared::sync_search`.
+2. Shared state resolves the session's filter ids against the host `FilterRegistry`, drops the in-flight operation, recompiles the filters, mints a new operation id, and returns `DropSearch` + `ApplySearchFilter`.
+3. The widget dispatches each through `UiActions`.
+4. `SessionService::handle_command` calls into `session_core`.
+5. Core callbacks are translated into `SessionMessage`s.
+6. The next `logic` applies them to `shared.search`; the next `ui` renders counts and pulls row text.
+
+## State Ownership
+
+- **UI-canonical:** `HostState`, `HostTabs`, `HostStorage`, and one `SessionShared` per session. These are the source of truth.
+- **Backend-canonical, mirrored in the UI:** log content, search results, bookmarks, attachments. Bookmarks are the clearest case: a toggle sends a command and only the resulting message mutates state. Never write these optimistically. This category exists because the current core owns that state, not by design; the intended direction is UI-canonical, so do not grow it.
+- **Derived caches:** compiled filters, table row caches, the published plugin snapshot. Invalidate on explicit state changes, never by recomputing in render.
+
+Persistence uses two separate stores: `host/ui/persist.rs` writes lightweight host preferences through eframe storage; everything heavier goes `HostStorage` → `HostCommand::SaveStorage` → the storage worker, driven by per-domain dirty flags.
+
+## Boundary to the Workspace
+
+`crates/stypes` holds the cross-crate contracts, `crates/core` is the engine, and `crates/app` owns presentation plus the *editable* config model (`ParserConfig`, `ByteSourceConfig`, `StreamConfig`). Those configs convert to `stypes` at a single point, `HostService::start_session`.
+
+Worth knowing: the config and storage types live under `host/ui/`, and the service layer imports them. `ui` is not a leaf layer — it is also the shared data-model module.
+
+Plugins follow the ownership rule strictly. `PluginsManager` exists only inside `PluginService`; the UI consumes cloned `PluginsState` snapshots and mutates through commands only. Plugin render metadata crosses as a `LogSchemaSpec`, resolved once by the host service and frozen into the session's `LogSchema` at creation, so an open session's columns cannot change when plugins reload.
+
+## Extension Points
+
+Ordered touch sequences:
+
+- **Parser:** `host/common/parsers.rs` (names and compatibility) → `host/ui/session_setup/state/parsers/` (`ParserConfig`) → side and main config forms → `HostService` conversion to `ParserType` and `LogSchemaSpec` → a new `session/ui/definitions/schema/` implementation.
+- **Source/transport:** `host/common/sources.rs` → parser compatibility → `session_setup/state/sources/` (`StreamConfig`) → main config form → `HostService` conversion to `ObserveOrigin`.
+- **Session operation:** `session/command.rs` → `session/message.rs` → `SessionService::handle_command` → `Session::handle_messages` → the sending widget.
+- **Host command:** `host/command.rs` → `HostService::handle_command` → optional `host/message.rs` → `Host::handle_message` → the sender.
+- **Shortcut:** no central enum; add a named field to `AppShortcuts` or `SessionShortcuts` and handle it. Deferred work needs a `ShortcutAction` variant.
+- **Palette command:** `CommandAction` variant → `scope()` → `execute_action` → the `COMMANDS` slice.
+- **Bottom tab:** `bottom_panel/tab_types.rs` → `render_content`, the tabs array, `bottom_tab_from_index`, and a field on `BottomPanelUI`.
+- **Storage domain:** `host/ui/storage/types.rs` → `HostStorage` collection and dirty tracking → `host/service/storage/` with a disk-IO submodule.
+
+## Traps
+
+These compile and fail at runtime or silently:
+
+- Mutating `SessionShared` public fields directly skips `bump_recent_revision`: the change renders but never persists into the recent session. Use the wrapper methods.
+- `shared.signals` is a render-frame queue. Emitting a signal from message handling or restore leaks it into the next frame; `debug_assert!`s guard this. Outside render use `sync_search_outcome`, not `sync_search`.
+- Compiled filter order must match the order sent in `ApplySearchFilter`, or match highlight colors land on the wrong filter.
+- A bottom tab added without extending `bottom_tab_from_index` panics at runtime instead of failing to compile.
+- A storage domain missing from dirty collection never persists; one missing from the failure re-mark never retries.
+- A palette command left out of the `COMMANDS` slice, or a shortcut left out of the definitions, is unreachable.

.ai/knowledge/crates/cli/AGENTS.md

@@ -41,7 +41,6 @@ It orchestrates core runtime crates and owns CLI argument parsing and output for
 
 ## Validation
 
-Run validation only when requested or needed to verify a change.
 Use targeted Cargo commands from the repository root, for example:
 
 - Check: `cargo check -p cli`

.ai/knowledge/crates/core/AGENTS.md

@@ -67,7 +67,6 @@ The native app and CLI depend on these crates directly through Rust APIs.
 
 ## Validation
 
-Run validation only when requested or needed to verify a change.
 Use targeted Cargo commands from the repository root, for example:
 
 - Check one crate: `cargo check -p <crate>`

.ai/knowledge/crates/stypes/AGENTS.md

@@ -39,7 +39,6 @@ Treat changes here as cross-crate API changes.
 
 ## Validation
 
-Run validation only when requested or needed to verify a change.
 Use targeted Cargo commands from the repository root, for example:
 
 - Check: `cargo check -p stypes`

.ai/knowledge/plugins/AGENTS.md

@@ -57,7 +57,6 @@ The plugin API crate is part of the main Rust workspace, while plugin examples a
 
 ## Validation
 
-Run validation only when requested or needed to verify a change.
 Use targeted commands for the area changed:
 
 - Main workspace plugin API: `cargo test -p plugins_api`

.ai/knowledge/rust-workspace.md

@@ -16,3 +16,12 @@ serde.workspace = true
 ```
 
 Keep the root workspace dependency list grouped by internal crates, external dependencies, and development dependencies.
+
+## Utility Crates
+
+Small, self-contained crates under `crates/`. Read the crate source directly; they need no separate context file.
+
+- `bufread`: buffered reader that refills while unread data is still buffered.
+- `dir_checksum`: recursive directory checksums honoring `gitignore` rules.
+- `file_tools`: file content classification, such as the UTF-8 text check.
+- `shell_tools`: discovery of installed shell profiles on the current platform.

.ai/rules/INDEX.md

@@ -0,0 +1,13 @@
+# Chipmunk Rules Index
+
+## Overview
+
+Repository code rules live under `.ai/rules/`.
+They are binding for every code change and every review in this repository, and they apply on top of the context in `.ai/knowledge/`.
+Use this file to choose the rule files that cover the areas being touched.
+A section marked `Red Flag` is a violation a review must report, even when the rest of the change is sound.
+
+## Rules
+
+- All Rust code: `.ai/rules/rust.md`
+- GUI application UI code: `.ai/rules/app-ui.md`

.ai/rules/app-ui.md

@@ -0,0 +1,54 @@
+# GUI Application UI Rules
+
+## Scope
+
+Applies to `egui` and `eframe` code, which lives in the GUI application crate `crates/app`.
+It applies on top of `.ai/rules/rust.md`, which holds for this code as well.
+
+## The Frame Loop Is Hot
+
+`update` and every `ui` path run on each frame. Keep out of them:
+
+- compute-heavy work,
+- per-frame allocation,
+- eager formatting of text that may never be shown,
+- derived values recomputed from scratch on every frame.
+
+Instead:
+
+- Move work off the UI thread when its result does not have to be recomputed every frame.
+- Cache reusable derived values such as validation results, filtered views, and formatted summaries, and invalidate them when their inputs change.
+
+## Red Flag: IO and Blocking Calls in the Render Loop
+
+Never perform IO in `update` or in any `ui` path: file reads and writes, directory scans, file metadata queries, network requests, or spawning processes. The same holds for every other blocking call, such as waiting on a channel or a lock, or driving a future with `block_on`.
+
+Each of these stalls the frame and freezes the window for the duration of the call. The cost stays invisible on a fast local machine with a warm cache, and appears on large files, slow or remote storage, and unreachable endpoints.
+
+Run the work in a host or session service instead, wrapping synchronous IO in `spawn_blocking` so the async runtime is not blocked either, and deliver the result to the UI as a `HostMessage` or `SessionMessage`. The UI renders from the state that message updated.
+
+Sending a message must also wake the UI, otherwise the result stays invisible until an unrelated frame happens to be drawn. `evaluate_send_res` in `crates/app/src/common/comm_utls.rs` requests the repaint on a successful send. Use it rather than a hand-rolled channel send, on the service side as well.
+
+The one sanctioned exception is grabbing log lines for the table being drawn, in `TableDelegate::prepare` of the logs and search tables. Those rows have to reach the screen in the frame that asks for them, so the grab blocks, with a retry and a timeout. Nothing else in the render loop may block.
+
+## Commands and Failures
+
+- Send commands from the UI through `UiActions::try_send_command` or `send_command_with_retry`, not through a channel sender directly. The helper logs the failure and raises a notification, while a raw send drops the user action silently.
+- Surface failures as an `AppNotification` together with a log entry. Widgets do not render error text inline. `Session::ok_or_notify` is the standard log-then-notify adapter.
+
+## Shared UI Code
+
+Before writing a UI building block, look for an existing one. Shared UI code lives in `crates/app/src/common/ui/`, `crates/app/src/host/common/`, and the `shared/` modules of each area. Reusing it keeps styling and behavior consistent across the app, while a local reimplementation drifts from the rest of it.
+
+Native file dialogs go through `actions.file_dialog` in `crates/app/src/host/ui/actions/file_dialog.rs`.
+
+## State Placement
+
+- Keep state placement explicit. In immediate mode nothing survives a frame unless it is stored deliberately.
+- Persist state that must outlive the frame instead of rebuilding it inside transient UI branches, unless resetting it is the intent.
+- UI state is the source of truth. A service owns backend runtime resources and publishes snapshots; it does not own state the UI renders from. Where the core keeps that state instead, mirror it read-only and let the resulting message be the only writer.
+
+## Tooltips
+
+- Use `Response::on_hover_ui(...)` instead of `on_hover_text(...)` when the content is computed or formatted, so it is built only on hover.
+- Inside `on_hover_ui(...)`, call `ui.set_max_width(ui.spacing().tooltip_width)` before adding content to preserve egui's default tooltip wrapping.