Skip to content
ai-nervPublic

About

A coding agent for Linux - a constellation of processes over Unix sockets, configured in Lua

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

magi

A coding agent for Linux.

How the four fit together — a turn end to end, writing a tool, memory both directions, what may run

Three computers had to agree before NERV moved. This is the one you talk to: the harness — a constellation of POSIX processes over Unix sockets, and a terminal that earns the name, with differential rendering into native scrollback, live streaming and an editor-grade prompt. Close the window and the turn keeps running.

The family

The MAGI of Neon Genesis Evangelion were three units deliberating, each running a facet of the same mind, and no one of them the whole. Here they are three programs, in three repositories, that talk over sockets and pipes:

magi this one: the harness — UI, host, providers, tools
melchior the agent layer — sessions talking to sessions, adoption, permissions
balthasar the memory layer — what was said, distilled and recalled
casper the tools, and the screen they draw on

They are separate programs, not components. melchior does not know what a harness is, and balthasar's Rust never parses magi's types — each one is useful, and testable, with the others absent. magi with no melchior is a session with no siblings, which is the ordinary case and not an error.

Status

A real agent: a model answers, tools run in their own processes, and the session is held by balthasar — which is also what makes it resumable, and what decides when it is compacted. magi keeps no transcript of its own. See PLAN.md for how it was built.

make run          # magi, for real, in the current directory
make install      # the static binary into $PREFIX/bin, so `magi` works anywhere
make configs      # install config/ into ~/.config/magi, ready to edit
make build        # the release binary, static where the toolchain allows it
make verify       # fmt, check, test, clippy, gates, docs

Both are needed. make configs installs configuration; it does nothing for a binary you have not installed. make configs says so when the two are out of step.

With no API keys set, make run says as much and /model lists every model magi knows with what each would need — so the first thing you do is choose one rather than read a config file.

Two backends, one renderer. alt takes the alternate screen and owns the transcript, which is what transcript search and selection will need; inline keeps a live region at the bottom of the normal screen and lets the terminal keep the history, so native scroll, search, and copy keep working. Both draw from the same components and answer the same keys, and the footer names whichever is active.

make run starts a daemon for the working directory and attaches the UI to it over a Unix socket — two processes, as the architecture intends. Quitting the UI detaches; the turn keeps running, and magi stop ends the daemon.

For working on the interface without a model, make demo replays a recorded session:

make demo         # the UI against a canned recording — no model, no tools
make host         # a replay host alone
make ui           # the UI alone, attaching to it

Tools

Four kinds, and the registry does not care which is which: a transport is a property of a declaration rather than a second registry, so each is checked against the same schema, asks the same person for the same permission, and is capped and masked on the way back.

Kind What it is
supplied from the tools program (casper) — one exec per call. Every tool that does anything to the machine
builtin compiled in: spawn — magi's only one, and coordination, not machine work
lua a function in the config's own VM — how the memory tools reach balthasar
command one exec per call, arguments built from the call — how agent reaches melchior

The machine tools all arrive from casper, which is another program and supplies the whole set. That makes it the largest trust assumption magi makes, and magi.casper_sha256 pins it to the bytes you set it up against. magi runs no tool of its own beyond spawn; there is no in-magi shell, no process peer, no MCP — running a tool is casper's.

casper is the default for the tools role, not a requirement: magi.tools = "workbench" hands the role to any program that answers ROLES.md's core, and magi.workbench_sha256 pins that one instead. Only your own configuration can name it — a project file that tries is refused.

magi tools        # what the model can call, and how each is reached
magi doctor       # what a session here would be made of, without starting one

magi doctor answers everything a session decides at start-up: which configuration was read, which of its lines were kept, what the registry ends up holding and where each entry came from, and whether the siblings are actually answering — asked, not looked for, because a program on $PATH is not a running one and a socket that accepts is not one that answers. It works on a machine where nothing is installed, which is the machine you run it on.

Memory

With balthasar running, a turn is shown what the project already remembers before it starts — without the model having to think to ask, which a model that has forgotten something cannot do. What is current is stated; what is merely on record is hedged, because balthasar decides which a memory is and flattening the two would present something you said once in March as true now.

It is bounded and on a clock: a tenth of the window, and 250ms to answer. A memory layer that is slow, wedged or absent costs the conversation nothing, which is what lets this be unconditional rather than a setting you have to find.

What the turn then does goes back — the only signal balthasar has for whether anything it offered was worth offering, and the axis MemoryArena separates from LoCoMo.

How this family talks

Three transports, two shapes, one encoding — written out because it was written out nowhere, and five wires had grown five ways to say the same thing.

Transport When Framing
argv a question with an answer and nothing to hold open one JSON object on stdout
pipe a parent and the child it started newline-delimited JSON, both directions
socket anything may knock four bytes of big-endian length, then JSON

JSON is on all three. It is the encoding, not a transport.

A call is answered; an event is not:

->  {"call":"status","args":[]}
<-  {"ok":true,"family":1,"n":1,"result":[{"busy":false}]}

    {"event":"listening","at":"…"}

result is a list and n says how long it is: a sibling that unpacked a bare value would read an answer as nothing at all. family says which revision the reply is written in — a reader refuses a number it does not know and tolerates one it predates. A refused call is a reply, not a dropped connection.

The tag key is event, everywhere, in both directions, and gate-wire refuses any other. The failure it prevents is silent: two of these wires exist as byte-identical copies in two repositories, so when two spellings drift nothing fails and no test goes red — the surface simply stops being answered.

Talking to other sessions

Concurrent clients

All attached clients share one session queue. Prompts, incoming messages, permission declarations and inherited grants are admitted in order. Prompts received during a turn wait for its completion and persistence attempt; they do not interrupt it. Consecutive incoming-agent messages at a boundary are read together. Up to 256 requests may wait; additional requests receive an explicit refusal instead of being silently dropped.

Interrupt requests cancellation of the active operation only. Queued prompts retain their order and receive fresh cancellation state. Model, provider, thinking, branch and resume changes are refused while work is active or queued. Disconnecting a client does not cancel work already accepted by the running host. The queue is in memory, not crash-durable; terminating the host loses requests that have not reached the transcript. A stopped worker produces a transcript error for each accepted prompt rather than leaving the queue stuck.

Idle resume flushes the current transcript, waits up to 30 seconds for its tracked helpers, and prepares the selected transcript before publishing a replacement snapshot. Failed storage, unsettled helpers, and missing or malformed replay leave the current session selected. A successful switch preserves attached clients, rebinds persistence and Lua/surface identity, and resets session-local layout, accounting and cancellation state. Stored cursors and child-transcript ownership survive both in-process and startup resume.

With melchior installed, a session can reach the other sessions in the same project. The model calls one agent tool — list, send, inbox, reply — and magi's whole knowledge of the layer is one file, magi-cli/src/melchior.rs, that spawns it and reads lines.

Two walls decide who can be reached. The project wall is the filesystem: another checkout's sessions are not refused, they are simply not there. The instance wall is the front door: a main speaks for its instance, and its subagents are private, so agent_talk is mains by default and instance or project when you mean otherwise.

A message carries a sort, and the sort decides what it may interrupt. question, answer, attention, trouble, handoff and report wake an idle session; only attention and trouble may reach one mid-turn. Anything arriving during a turn waits. Consecutive arrivals are committed together at their queue boundary; they start one reply only when at least one has a waking sort. Notes alone do not start a reply.

One main may ask another to adopt it. Consent is a person's: the request surfaces as a prompt on the other side, and accepting hands down exactly the grants the parent already holds — never more. A child that wants something outside them is refused and told to ask its parent.

A session can also be started under another. melchior fork mints a name and a secret; the harness spawns with what it was handed, and the session that comes up is a child — it writes the note that makes the tree readable off the directory, it is inside the walls the policy draws, and the session that minted its secret is the only one that can end it. melchior names; the harness spawns, because a layer that started harnesses would have to know what one is.

spawn returns a child agent id. Check it with agent verb status, who set to that id, or read crew. task instead takes the exact id@message-id handle returned by ask in about. A tool-call refusal is not a child's failure report, and a report arriving while its phase is working does not mean it has finished. Deferred finish/blocked wakes are canceled if the agent resumes before delivery; a wake still asks the coordinator to check current status. A headless coordinator reports waiting, not finished, while a result-collection wake is queued.

Every child turn hands in an end-of-turn report, including an empty answer, interruption, provider failure or stopped worker. The harness preserves an explicitly submitted report, adds final-answer and error information, and labels the outcome. This describes a turn, not completion of delegated children. Workers must still use agent verb report for their findings; they must not substitute send.

Report notifications queue behind active work. Before the parent's next model call, the harness loads the stored report into the conversation and verifies its content digest. Submission revisions deduplicate notifications, not equal text from separate turns. Automatic loading carries at most 40,000 UTF-8 bytes per report; a longer report includes an explicit paged-read instruction. Escape cancels the current turn only: reports arriving afterward wake it again, and an interrupted report-handling turn is retried without another notification. Pending receipts last for the live session; forced process termination or unavailable coordination can still prevent delivery. These hooks do not reconstruct pending receipts after a restart.

Layout

Crate Role
magi-proto the wire contract: events, commands, envelope. No I/O
magi-ipc Unix socket transport, length-prefixed CBOR, SO_PEERCRED identity
magi-model the provider-neutral message model
magi-provider the HTTP side: streaming, SSE, retries, what each error means
magi-core the turn loop, as an explicit state machine
magi-tools what a tool is, and how one is dispatched to the tools program
magi-lua the Lua VM, and the config API it offers init.lua
magi-journal the transcript this process holds; balthasar is what stores it
magi-host the session: the transcript, the socket, and the turns
magi-tui rendering: theme, markdown, transcript, editor, status, footer
magi-cli the UI process, and melchior.rs — everything magi knows of the agent layer
magi-testkit fake harness and recordings

Development

.make.lua is the task interface; make on its own lists every recipe.

make test         # the suite
make gates        # the architectural gates
make clippy       # warnings denied
make verify       # all of it
Gate Rule
gate-file-size no .rs over 800 lines
gate-modules every .rs is reachable from its crate root
gate-proto-size magi-proto under 4,000 lines
gate-reachable no crate unreachable from the binary
gate-cycles no two top-level modules depend on each other
gate-hermetic the suite leaves behind no file and no process
gate-wire one way of saying a thing crosses a boundary
gate-one-store balthasar holds the history and magi keeps no copy
gate-family the binary answers the family contract
gate-sandbox one Lua VM, sandboxed, and a checkout cannot govern the session
gate-lints every crate takes the workspace's denials and nothing takes them back
gate-comments a comment describes the block; it is not a fifth of the code arguing with it

gate-cycles is the one pi never built. It built reachability — and a cycle is maximally reachable, so a reachability gate passes at 240,000 lines with the knot still in it. Ours had the same blind spot, and this landed with an empty allowlist in all four repositories after finding three cycles nobody had named.

gate-hermetic runs the suite under a TMPDIR of its own and asserts it is empty afterwards. Every test used to tidy up on its last line — and assert! unwinds straight past a trailing remove_dir_all, so a failing test always leaked. Three thousand six hundred directories had collected before anything looked. It also counts what survives: a leaked process is found by its working directory or by $TMPDIR appearing in its environment, because cargo runs a test binary in the package directory and a child inherits that — the cwd question alone is blind to every test that does not call current_dir.

gate-sandbox holds the count of Lua VMs at one. The sandbox's own tests prove the removals work; what they cannot prove is that every VM went through the constructor that applies them, and a second Lua::full() is a full standard library with no test anywhere going red.

gate-modules earns its place on its own: a file nobody declares is not a compile error, not a warning and not run — it simply is not part of the crate. Two were found at once, each holding tests that had silently not run since the commit that moved them.

gate-comments caps comment lines at 20% of the code in the same file. SAFETY: notes do not count, from the SAFETY: line onward, and every file gets a floor of three lines whatever its size. It flagged 175 files at first: the prose had grown into the argument for each decision, the story of the bug behind it, and the case against alternatives nobody had proposed. What a comment is for is the block underneath it.

The gates are not advisory. Every agent this one was measured against carries dead code nothing reaches and a god file in the tens of thousands of lines — 6,549 in one, 34,875 in another. None of that was decided; it arrived one reasonable commit at a time, which is the only way it ever arrives, and the reason the limit has to be a script rather than an intention.

Releases

make dist builds against x86_64-unknown-linux-musl, which produces a genuine static-pie binary — no interpreter, no NEEDED entries, nothing to install alongside it:

magi 0.1.0   2.15 MB
binary  target/x86_64-unknown-linux-musl/release/magi
size    2.15 MB   2,249,400 bytes
linking ✓ static   no runtime dependencies

The linkage line is read out of the ELF, not inferred from the flags: a glibc "static" build still carries an INTERP and dies on a machine whose loader disagrees, and ldd reports it as statically linked anyway.

The release profile is pinned rather than left to defaults — lto = "thin", codegen-units = 16 — because a build that silently switches to fat LTO and one codegen unit turns a ten-second link into minutes.

Tagging a release builds the same binary on the runner, for amd64 and arm64, and attaches it. The toolchain is the flake's, so a release is built with the compiler the gates ran under.

Configuration

Everything magi knows about the outside world is Lua, and it all lives in config/:

config/apis/*.lua the wire protocols — how to talk to an endpoint
config/providers.lua the catalog — which endpoints exist and what they offer
config/tools.lua what the model may call, and how each tool is reached
config/clients/*.lua a sibling's client library, if you name one — none ships, because client serves it
config/init.lua your settings, and anything you want to add

make configs copies them to $XDG_CONFIG_HOME/magi/, where magi reads them. The binary also carries a copy, so a fresh install already speaks and already has a catalog — installing gives you the real files to edit, it does not turn anything on that was off.

Layered, later winning by registration id:

compiled-in defaults  →  ~/.config/magi/apis/*.lua  →  providers.lua  →  init.lua  →  ./.magi.lua

A provider or a protocol declared twice replaces rather than appends, which is what makes both an override and a loop over a directory of machines safe to re-run. A file that exists and does not load is fatal: it expressed an intention that has not been carried out.

Model discovery

:model opens the picker and refreshes provider catalogs through Melchior, bypassing its discovery cache. The list updates without restarting the session or changing the active model. Failed providers keep their previous choices with a warning; successful empty catalogs remove models that are no longer available. Closing the picker does not reopen it when refresh finishes. Magi and Melchior both need builds supporting this refresh protocol.

About

A coding agent for Linux - a constellation of processes over Unix sockets, configured in Lua

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages