diff --git a/README.md b/README.md index 946bdc32..b2412db6 100644 --- a/README.md +++ b/README.md @@ -1,106 +1,268 @@ # Gas Town -> **Status**: Experimental (v0.1) - We're exploring these ideas and invite you to explore with us. +Multi-agent orchestrator for Claude Code. Sling work to agents; they run it. -Gas Town is an experiment in multi-agent coordination for Claude Code. It provides infrastructure for spawning workers, tracking work via molecules, and coordinating merges. +## Why Gas Town? -We think of it using steam-age metaphors: - -``` -Claude = Fire (the energy source) -Claude Code = Steam Engine (harnesses the fire) -Gas Town = Steam Train (coordinates engines on tracks) -Beads = Railroad Tracks (the persistent ledger) -``` - -The goal is a "village" architecture - not rigid hierarchy, but distributed awareness where agents can help neighbors when something is stuck. Whether this actually works at scale is something we're still discovering. - -## Prerequisites - -- **Go 1.23+** - For building from source -- **Git** - For rig management and beads sync -- **tmux** - Required for agent sessions (all workers run in tmux panes) -- **Claude Code CLI** - Required for agents (`claude` command must be available) - -## Install - -**From source (recommended for now):** - -```bash -go install github.com/steveyegge/gastown/cmd/gt@latest -``` - -**Package managers (coming soon):** - -```bash -# Homebrew (macOS/Linux) -brew install gastown - -# npm (cross-platform) -npm install -g @anthropic/gastown -``` +| Without | With Gas Town | +|---------|---------------| +| Agents forget work after restart | Work persists on hooks - survives crashes, compaction, restarts | +| Manual coordination | Agents have mailboxes, identities, and structured handoffs | +| 4-10 agents is chaotic | Comfortably scale to 20-30 agents | +| Work state in agent memory | Work state in Beads (git-backed ledger) | ## Quick Start ```bash -# Create a town (workspace) +# Install +go install github.com/steveyegge/gastown/cmd/gt@latest + +# Create workspace gt install ~/gt -# Add a project rig -gt rig add myproject --remote=https://github.com/you/myproject.git +# Add a project +gt rig add myproject --remote=https://github.com/you/repo.git -# Assign work to a polecat -gt sling myproject-123 myproject +# Sling work to a polecat (worker) +gt sling issue-123 myproject ``` -## Architecture +## Core Concepts ``` -Town (~/gt/) -├── Mayor (global coordinator) -└── Rig: myproject - ├── Witness (lifecycle manager) - ├── Refinery (merge queue) - └── Polecats (workers) +Town (~/gt/) Your workspace +├── Rig (project) Container for a git project + its agents +│ ├── Polecats Workers (ephemeral, spawn → work → disappear) +│ ├── Witness Monitors workers, handles lifecycle +│ └── Refinery Merge queue processor +└── Mayor Global coordinator ``` -## Key Concepts +**Hook**: Each agent has a hook where work hangs. On wake, run what's on your hook. -Ideas we're exploring: +**Beads**: Git-backed issue tracker. All work state lives here. [github.com/steveyegge/beads](https://github.com/steveyegge/beads) -- **Molecular Chemistry of Work**: Protos (templates) → Mols (flowing work) → Wisps (ephemeral) → Digests (outcomes) -- **Beads**: Git-backed, human-readable ledger for tracking work ([github.com/steveyegge/beads](https://github.com/steveyegge/beads)) -- **Village Model**: Distributed awareness instead of centralized monitoring -- **Propulsion Principle**: Agents pull work from molecules rather than waiting for commands -- **Nondeterministic Idempotence**: The idea that any worker can continue any molecule after crashes +## Workflows -Some of these are implemented; others are still aspirational. See docs for current status. +### Minimal (No Tmux) +Run individual Claude Code instances manually. Gas Town just tracks state. +```bash +gt sling issue-123 myproject # Creates work assignment +claude --resume # Agent reads mail, runs work +``` -## Commands +### Full Stack (Tmux) +Agents run in tmux sessions. Daemon manages lifecycle. +```bash +gt daemon start # Start lifecycle manager +gt sling issue-123 myproject # Spawns polecat automatically +``` + +### Pick Your Roles +Gas Town is modular. Run what you need: +- **Polecats only**: Manual spawning, no monitoring +- **+ Witness**: Automatic worker lifecycle, stuck detection +- **+ Refinery**: Merge queue, code review +- **+ Mayor**: Cross-project coordination + +## Cooking Formulas + +Formulas define structured workflows. Cook them, sling them to agents. + +### Basic Example + +```toml +# .beads/formulas/shiny.formula.toml +formula = "shiny" +description = "Design before code, review before ship" + +[[steps]] +id = "design" +description = "Think about architecture" + +[[steps]] +id = "implement" +needs = ["design"] + +[[steps]] +id = "test" +needs = ["implement"] + +[[steps]] +id = "submit" +needs = ["test"] +``` + +### Using Formulas ```bash -gt status # Town status -gt rig list # List rigs -gt sling # Assign work to polecat -gt mail inbox # Check messages -gt peek # Check worker health -gt nudge # Wake stuck worker +bd formula list # See available formulas +bd cook shiny # Cook into a protomolecule +bd pour shiny --var feature=auth # Create runnable molecule +gt sling gt-xyz myproject # Assign to worker ``` -## Documentation +### What Happens -- [Vision](docs/vision.md) - Core innovations and philosophy -- [Architecture](docs/architecture.md) - System design -- [Molecular Chemistry](docs/molecular-chemistry.md) - Work composition -- [Molecules](docs/molecules.md) - Workflow templates +1. **Cook** expands the formula into a protomolecule (frozen template) +2. **Pour** creates a molecule (live workflow) with steps as beads +3. **Worker executes** each step, closing beads as it goes +4. **Crash recovery**: Worker restarts, reads molecule, continues from last step -## Development +### Example: Beads Release Molecule + +A real workflow for releasing a new beads version: + +```toml +formula = "beads-release" +description = "Version bump and release workflow" + +[[steps]] +id = "bump-version" +description = "Update version in version.go and CHANGELOG" + +[[steps]] +id = "update-deps" +needs = ["bump-version"] +description = "Run go mod tidy, update go.sum" + +[[steps]] +id = "run-tests" +needs = ["update-deps"] +description = "Full test suite, check for regressions" + +[[steps]] +id = "build-binaries" +needs = ["run-tests"] +description = "Cross-compile for all platforms" + +[[steps]] +id = "create-tag" +needs = ["build-binaries"] +description = "Git tag with version, push to origin" + +[[steps]] +id = "publish-release" +needs = ["create-tag"] +description = "Create GitHub release with binaries" +``` + +Cook it, pour it, sling it. The polecat runs through each step, and if it crashes +after `run-tests`, a new polecat picks up at `build-binaries`. + +### Formula Composition + +```toml +# Extend an existing formula +formula = "shiny-enterprise" +extends = ["shiny"] + +[compose] +aspects = ["security-audit"] # Add cross-cutting concerns +``` + +## Key Commands + +### For Humans (Overseer) ```bash -go build -o gt ./cmd/gt -go test ./... +gt start # Start Gas Town (daemon + agents) +gt shutdown # Graceful shutdown +gt status # Town overview +gt attach # Jump into any agent session + # e.g., gt mayor attach, gt witness attach ``` +Most other work happens through agents - just ask them. + +### For Agents + +```bash +# Work +gt sling # Assign work to polecat +bd ready # Show available work +bd list --status=in_progress # Active work + +# Communication +gt mail inbox # Check messages +gt mail send -s "..." -m "..." + +# Lifecycle +gt handoff # Request session cycle +gt peek # Check agent health + +# Diagnostics +gt doctor # Health check +gt doctor --fix # Auto-repair +``` + +## Roles + +| Role | Scope | Job | +|------|-------|-----| +| **Overseer** | Human | Sets strategy, reviews output, handles escalations | +| **Mayor** | Town-wide | Cross-rig coordination, work dispatch | +| **Deacon** | Town-wide | Daemon process, agent lifecycle, plugin execution | +| **Witness** | Per-rig | Monitor polecats, nudge stuck workers | +| **Refinery** | Per-rig | Merge queue, PR review, integration | +| **Polecat** | Per-task | Execute work, file discovered issues, request shutdown | + +## The Propulsion Principle + +> If your hook has work, RUN IT. + +Agents wake up, check their hook, execute the molecule. No waiting for commands. +Molecules survive crashes - any agent can continue where another left off. + +--- + +## Optional: MEOW Deep Dive + +**M**olecular **E**xpression **O**f **W**ork - the full algebra. + +### States of Matter + +| Phase | Name | Storage | Behavior | +|-------|------|---------|----------| +| Ice-9 | Formula | `.beads/formulas/` | Source template, composable | +| Solid | Protomolecule | `.beads/` | Frozen template, reusable | +| Liquid | Mol | `.beads/` | Flowing work, persistent | +| Vapor | Wisp | `.beads/` (ephemeral flag) | Transient, for patrols | + +*(Protomolecules are an homage to The Expanse. Ice-9 is a nod to Vonnegut.)* + +### Operators + +| Operator | From → To | Effect | +|----------|-----------|--------| +| `cook` | Formula → Protomolecule | Expand macros, flatten | +| `pour` | Proto → Mol | Instantiate as persistent | +| `wisp` | Proto → Wisp | Instantiate as ephemeral | +| `squash` | Mol/Wisp → Digest | Condense to permanent record | +| `burn` | Wisp → ∅ | Discard without record | + +### Dynamic Bonding + +Patrol workflows can spawn children at runtime: + +```bash +# Witness discovers 3 polecats, bonds an arm for each +for polecat in ace nux toast; do + bd mol bond mol-polecat-arm $PATROL_ID --var name=$polecat +done +``` + +Creates a "Christmas ornament" shape - trunk with dynamic arms. + +--- + +## Prerequisites + +- **Go 1.23+** +- **Git** +- **tmux** (for full stack mode) +- **Claude Code CLI** + ## License MIT diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 8e848273..00000000 --- a/docs/architecture.md +++ /dev/null @@ -1,2341 +0,0 @@ -# Gas Town Architecture - -Gas Town is a multi-agent workspace manager that coordinates AI coding agents working on software projects. It provides the infrastructure for spawning workers, processing work through a priority queue, and coordinating agents through mail and issue tracking. - -**Gas Town is a stream engine, not a swarm engine.** Polecats (workers) can start and land at any time, independently. They're ephemeral - once their work merges, they evaporate. You can spawn a batch of polecats to tackle parallelizable work, but there's no "swarm ID" or batch boundary. The merge queue optionally serializes landing, preventing monkey-knife-fights when parallel polecats touch the same code. A single polecat is just a batch of one. - -**Key insight**: Work is a stream, not discrete batches. The Refinery's merge queue is the coordination mechanism. Beads (issues) are the data plane. There are no "swarm IDs" - just epics with children, processed by workers, merged through the queue. - -**Molecule-first paradigm**: Gas Town is fundamentally a molecule execution engine. Workers don't just "work on issues" - they execute molecules. The issue is seed data; the molecule defines the workflow. This enables nondeterministic idempotence: any worker can pick up where another left off, surviving crashes, context compaction, and restarts. If a process requires cognition, it should be a molecule. See [Molecules](#molecules-composable-workflow-templates) for full details. - -**The Steam Engine Metaphor**: Gas Town is an engine. Engines do work and generate steam. In our system: -- **Proto molecules** are the fuel (templates that define workflows) -- **Wisps** are the steam (transient execution traces that rise and dissipate) -- **Digests** are the distillate (condensed permanent records of completed work) - -Just as steam can dissipate or be condensed into useful output, wisps can be burned (cleaned up) or squashed (compressed into digests). This metaphor runs through all of Gas Town's vocabulary: bond, burn, squash, wisp. - -## System Overview - -```mermaid -graph TB - subgraph "Gas Town" - Overseer["👤 Overseer
(Human Operator)"] - - subgraph Town["Town (~/gt/)"] - Mayor["🎩 Mayor
(Global Coordinator)"] - - subgraph Rig1["Rig: wyvern"] - W1["👁 Witness"] - R1["🔧 Refinery"] - P1["🐱 Polecat"] - P2["🐱 Polecat"] - P3["🐱 Polecat"] - end - - subgraph Rig2["Rig: beads"] - W2["👁 Witness"] - R2["🔧 Refinery"] - P4["🐱 Polecat"] - end - end - end - - Overseer --> Mayor - Mayor --> W1 - Mayor --> W2 - W1 --> P1 - W1 --> P2 - W1 --> P3 - W2 --> P4 - P1 -.-> R1 - P2 -.-> R1 - P3 -.-> R1 - P4 -.-> R2 -``` - -## Core Concepts - -### HQ (Town) - -The **HQ** (headquarters) is the installation directory where Gas Town lives - the physical root of your workspace. The terms "HQ" and "town" are often used interchangeably: -- **HQ** = physical (the directory at `~/gt/`) -- **Town** = logical (the Gas Town workspace concept) - -An HQ contains: -- `CLAUDE.md` - Mayor role context (Mayor runs from HQ root) -- `mayor/` - Mayor configuration, state, and registry -- `.beads/` - Town-level beads (hq-* prefix for mayor mail) -- `rigs/` or rig directories - Managed project containers - -Create an HQ with `gt install`: -```bash -gt install ~/gt --git # Create HQ with git -``` - -**See**: [docs/hq.md](hq.md) for comprehensive HQ documentation, including: -- Beads redirect patterns for complex setups -- HQ templates for organizations -- Migration between HQs - -### Rig - -A **Rig** is a container directory for managing a project and its agents. Importantly, the rig itself is NOT a git clone - it's a pure container that holds: -- Rig configuration (`config.json`) -- Rig-level beads database (`.beads/`) for coordinating work -- Agent directories, each with their own git clone - -This design prevents agent confusion: each agent has exactly one place to work (their own clone), with no ambiguous "rig root" that could tempt a lost agent. - -### Overseer (Human Operator) - -The **Overseer** is the human operator of Gas Town - not an AI agent, but the person who runs the system. The Overseer: - -- **Sets strategy**: Defines project goals and priorities -- **Provisions resources**: Adds machines, polecats, and rigs -- **Reviews output**: Approves merged code and completed work -- **Handles escalations**: Makes final decisions on stuck or ambiguous work -- **Operates the system**: Runs `gt` commands, monitors dashboards - -The Mayor reports to the Overseer. When agents can't resolve issues, they escalate up through the chain: Polecat → Witness → Mayor → Overseer. - -### Agents - -Gas Town has four AI agent roles: - -| Agent | Scope | Responsibility | -|-------|-------|----------------| -| **Mayor** | Town-wide | Global coordination, work dispatch, cross-rig decisions | -| **Witness** | Per-rig | Worker lifecycle, nudging, pre-kill verification, session cycling | -| **Refinery** | Per-rig | Merge queue processing, PR review, integration | -| **Polecat** | Per-rig | Implementation work on assigned issues | - -### Village Architecture - -We're experimenting with a "village" model for agent coordination. - -The pattern we're trying to avoid: -``` -Centralized Monitor → watches all workers → single point of failure - → fragile protocols → cascading failures -``` - -The pattern we're exploring: -``` -Every worker → understands the whole → can help any neighbor - → peek is encouraged → distributed awareness -``` - -**Properties we're building toward:** - -- **Distributed awareness**: Agents understand the system, not just their task -- **Mutual monitoring**: Any agent can peek at any other agent's health -- **Collective intervention**: If you see something stuck, you can help - -Whether this achieves actual resilience at scale is still being tested. - -**Practical implications:** - -1. **Every patrol includes neighbor-checking** - Witness peeks at Refinery, Refinery peeks at Witness, everyone can peek at the Deacon -2. **`gt peek` is universal vocabulary** - Any agent can check any other agent's health -3. **Exit state enums are teaching tools** - COMPLETED, BLOCKED, REFACTOR, ESCALATE are shared vocabulary -4. **Mail is the nervous system** - Asynchronous, persistent, auditable coordination - -### Mail - -Agents communicate via **mail** - messages stored as beads issues with `type=message`. Mail enables: -- Work assignment (Mayor → Refinery → Polecat) -- Status reporting (Polecat → Witness → Mayor) -- Session handoff (Agent → Self for context cycling) -- Escalation (Witness → Mayor for stuck workers) - -**Two-tier mail architecture:** -- **Town beads** (prefix: `gm-`): Mayor inbox, cross-rig coordination, handoffs -- **Rig beads** (prefix: varies): Rig-local agent communication - -Mail commands use beads issues with type=message: -```bash -gt mail send mayor/ -s "Subject" -m "Body" # Creates message issue -gt mail inbox # Lists message issues -gt mail read gm-abc # Shows message issue -``` - -```mermaid -flowchart LR - subgraph "Communication Flows" - direction LR - Mayor -->|"dispatch work"| Refinery - Refinery -->|"assign issue"| Polecat - Polecat -->|"done signal"| Witness - Witness -->|"work complete"| Mayor - Witness -->|"escalation"| Mayor - Mayor -->|"escalation"| Overseer["👤 Overseer"] - end -``` - -### Beads - -**Beads** is the issue tracking system. Gas Town agents use beads to: -- Track work items (`bd ready`, `bd list`) -- Create issues for discovered work (`bd create`) -- Claim and complete work (`bd update`, `bd close`) -- Sync state to git (`bd sync`) - -Polecats have direct beads write access and file their own issues. - -#### Beads Configuration for Multi-Agent - -Gas Town uses beads in a **shared database** configuration where all agents in a rig share one `.beads/` directory. This requires careful configuration: - -| Agent Type | BEADS_DIR | BEADS_NO_DAEMON | sync-branch | Notes | -|------------|-----------|-----------------|-------------|-------| -| Polecat (worktree) | rig/.beads | **YES (required)** | recommended | Daemon can't handle worktrees | -| Polecat (full clone) | rig/.beads | Optional | recommended | Daemon safe but sync-branch helps | -| Refinery | rig/.beads | No | optional | Owns main, daemon is fine | -| Witness | rig/.beads | No | optional | Read-mostly access | -| Mayor | rig/.beads | No | optional | Infrequent access | - -**Critical: Worktrees require no-daemon mode.** The beads daemon doesn't know which branch each worktree has checked out, and can commit/push to the wrong branch. - -**Environment setup when spawning agents:** - -```bash -# For worktree polecats (REQUIRED) -export BEADS_DIR=/path/to/rig/.beads -export BEADS_NO_DAEMON=1 - -# For full-clone polecats (recommended) -export BEADS_DIR=/path/to/rig/.beads -# Daemon is safe, but consider sync-branch for coordination - -# Rig beads config.yaml should include: -sync-branch: beads-sync # Separate branch for beads commits -``` - -**Why sync-branch?** When multiple agents share a beads database, using a dedicated sync branch prevents beads commits from interleaving with code commits on feature branches. - -#### Beads as Data Plane (CLAW) - -Gas Town uses Beads for persistence - **C**ommits as **L**edger of **A**ll **W**ork. -We've combined the data plane and control plane: -- Traditional systems separate "what's stored" from "what to do next" -- We store both in beads, so agent state survives crashes - -In traditional systems: -- **Data plane**: Stores information (issues, messages) -- **Control plane**: Coordinates behavior (what to do next, who does what) - -In Gas Town, **the control state IS data in beads**. Molecule steps, dependencies, and status ARE the control plane. Agents read beads to know what to do next. - -This intentional blur provides: -- **Fault tolerance**: Control state survives agent crashes (it's in beads, not agent memory) -- **Observability**: `bd list` shows the full system state -- **Decentralization**: Each agent reads its own state from beads -- **Recovery**: Restart = re-read beads = continue from where you left off - -There is no separate orchestrator maintaining workflow state. Beads IS the orchestrator. - -| Category | Description | Status | -|----------|-------------|--------| -| **Work items** | Issues, tasks, epics | Core | -| **Mail** | Messages between agents (`type: message`) | Core | -| **Merge requests** | Queue entries (`type: merge-request`) | In progress | -| **Molecules** | Composable workflow templates | Planned (v1) | -| **Timed beads** | Scheduled recurring work | Planned (post-v1) | -| **Pinned beads** | Ongoing concerns that don't close | Planned (post-v1) | -| **Resource beads** | Leases, locks, quotas | Planned (post-v1) | - -#### Two-Level Beads Architecture - -Gas Town uses a **two-level beads architecture**. This is critical to understand: - -``` -~/gt/ # Town repo (stevey-gt.git) -├── .beads/ # TOWN-LEVEL: HQ beads (tracked here) -│ ├── config.yaml # NO sync-branch (single clone) -│ └── issues.jsonl # hq-* prefix for mayor mail -│ -└── gastown/ # Rig container (NOT a git clone) - ├── .beads/ # GITIGNORED - local runtime state - │ └── (populated at runtime) - │ - └── crew/max/ # Project repo clone (gastown.git) - └── .beads/ # RIG-LEVEL: Project beads (tracked in gastown.git) - ├── config.yaml # sync-branch: beads-sync - └── issues.jsonl # gt-* prefix for project issues -``` - -**Key points:** - -| Level | Location | Git Repo | sync-branch | Prefix | Purpose | -|-------|----------|----------|-------------|--------|---------| -| Town | `~/gt/.beads/` | stevey-gt.git | NOT set | `hq-*` | Mayor mail, cross-rig coordination | -| Rig | `~/gt/gastown/crew/max/.beads/` | gastown.git | `beads-sync` | `gt-*` | Project bugs, features, tasks | - -**Why two levels?** -- **Town beads** are for the Gas Town installation itself (only one clone) -- **Rig beads** are for the project (shared across multiple clones: crew, polecats, mayor/rig) - -**Why different sync-branch settings?** -- **Town beads**: Single clone at HQ, no coordination needed, commits to main -- **Rig beads**: Multiple clones (polecats, crew, refinery), need `beads-sync` branch to avoid conflicts with code commits - -**Common confusion:** -- `~/gt/gastown/.beads/` at the rig container level is **gitignored** (local runtime state) -- The real project beads live in the **gastown.git worktrees** (e.g., `crew/max/.beads/`) -- All clones share the same beads via git sync on the `beads-sync` branch - -#### Mail Routing - -Mail is routed to the correct beads database based on recipient address. The `Router` (in `internal/mail/router.go`) handles this: - -``` -Sender → Router.Send() → resolveBeadsDir(recipient) → creates message issue in target beads -``` - -**Routing logic (`resolveBeadsDir`):** - -| Recipient | Beads Location | Example | -|-----------|----------------|---------| -| Town-level (`mayor/`, `deacon/`) | `{townRoot}/.beads` | `~/gt/.beads` | -| Rig-level (`rig/polecat`) | `{townRoot}/{rig}/.beads` | `~/gt/gastown/.beads` | -| Unknown/fallback | Town-level beads | `~/gt/.beads` | - -**Town root detection:** -The router finds the town root by walking up directories looking for `mayor/town.json`. If not found, it falls back to the caller's workDir. - -**Environment setup:** -All `bd` commands are invoked with: -- `BEADS_DIR=` - Routes to correct database -- `BEADS_AGENT_NAME=` - Identifies sender - -#### Shared Beads for Polecats - -Polecats use **redirect files** instead of their own beads databases. This eliminates git sync overhead between polecat worktrees. - -**Structure:** -``` -rig/ - .beads/ ← Shared database (rig-level) - polecats/ - / - .beads/ - redirect ← Contains "../../.beads" -``` - -**How it works:** -1. When a polecat is spawned, `setupSharedBeads()` (in `internal/polecat/manager.go`) creates the redirect file -2. The beads CLI reads the redirect file and follows it to the rig's shared database -3. All polecats read/write the same beads database - no git sync needed - -**Benefits:** -- No JSONL merge conflicts between polecats -- Instant visibility of issue updates across all workers -- Reduced git operations (no beads-sync branch coordination for polecats) - -**Redirect vs Clone beads:** - -| Agent Type | Beads Location | Method | -|------------|----------------|--------| -| Polecat (worktree) | Redirect to `rig/.beads` | `.beads/redirect` file | -| Crew worker (clone) | Own `.beads/` | Git sync on `beads-sync` | -| Mayor/Refinery | Rig's `.beads/` | Direct or symlink | - -**Molecules** are crystallized workflow patterns that can be attached to work items. See the dedicated **Molecules** section below for full details on composition, nondeterministic idempotence, and built-in workflows. - -**The OS Metaphor**: Gas Town is an operating system for work: - -| OS Concept | Gas Town | -|------------|----------| -| Kernel | Daemon | -| Process scheduler | Ready work + dependencies | -| Timer interrupts | Timed beads | -| Semaphores | Resource beads | -| Background services | Pinned beads | -| Process templates | Proto molecules | -| Running processes | Wisp molecules | -| Process termination | Burn (discard) or squash (save state) | -| IPC | Mail beads | - -## Molecules: Composable Workflow Templates (MEOW) - -**MEOW** - **M**olecular **E**xpression **O**f **W**ork. Molecules are **crystallized, -composable, nondeterministic-idempotent workflow templates**. They encode structured -workflows that any worker can execute, with full auditability and the ability for -any worker to pick up where another left off. - -Work flows through the **Rig → Cook → Run** lifecycle: -- **Rig**: Compose workflow formulas (TOML/JSON files with `extends`, `compose`) -- **Cook**: Transform formulas into executable protos (`bd cook` expands macros, applies aspects) -- **Run**: Agents execute the cooked workflow (pour creates mols, wisp creates ephemeral traces) - -See [molecular-chemistry.md](molecular-chemistry.md) for the full lifecycle specification. - -### Core Concepts - -| Concept | Name | Description | -|---------|------|-------------| -| Source | **Formula** | TOML/JSON workflow definition with composition rules (.formula.toml preferred) | -| Template | **Proto Molecule** | Cooked workflow pattern (the "fuel") | -| Running execution | **Wisp Molecule** | Transient execution trace (the "steam") | -| Permanent record | **Digest** | Compressed summary of completed work (the "distillate") | -| Individual step | **Atom/Step** | Smallest unit of work within a molecule | -| Dependency | **Bond** | Connection between steps (Needs: directive); also the act of instantiation | -| Composed molecule | **Polymer/Derived** | Molecule built from other molecules | -| Discard execution | **Burn** | Delete wisps without saving (routine work) | -| Compress execution | **Squash** | Compress wisps into a digest (preserve outcome) | - -### Molecule Phase Lifecycle - -Molecules follow a **states of matter** metaphor through their lifecycle: - -``` - ┌─────────────┐ - │ Proto │ - │ (crystal) │ - └──────┬──────┘ - │ - bd mol bond - │ - ┌────────────┴────────────┐ - │ │ - ▼ ▼ - ┌───────────────┐ ┌───────────────┐ - │ Mol │ │ Wisp │ - │ (liquid) │ │ (gas) │ - │ durable │ │ transient │ - │ main beads │ │ .beads-wisp/ │ - └───────┬───────┘ └───────┬───────┘ - │ │ - bd mol squash bd mol squash - │ │ - ▼ ▼ - ┌───────────────┐ ┌───────────────┐ - │ Digest │ │ (nothing) │ - │ (distillate) │ │ evaporates │ - │ in git hist │ └───────────────┘ - └───────────────┘ -``` - -**Phase transitions:** -- **Proto → Mol/Wisp** (`bd mol bond`): Instantiate a template into a running execution -- **Mol → Digest** (`bd mol squash`): Compress completed work into permanent record -- **Wisp → (evaporates)** (`bd mol squash` or `bd mol burn`): Transient trace disappears - -### When to Use Mol vs Wisp - -The choice between **Mol** (durable) and **Wisp** (transient) depends on the work's importance and audit requirements: - -| Aspect | Mol (Durable) | Wisp (Transient) | -|--------|---------------|------------------| -| **Storage** | Main `.beads/` database | `.beads-wisp/` directory | -| **Persistence** | Survives indefinitely | Evaporates on squash/burn | -| **Git tracking** | Committed, synced | Never committed | -| **Audit trail** | Full history preserved | Only digest (if squashed) | -| **Use case** | Important work | Routine operations | - -**Use Mol for:** -- Code review waves (need audit trail of findings) -- Epic implementation (track progress across sessions) -- Feature work (preserve design decisions) -- Anything you might need to reference later - -**Use Wisp for:** -- Orchestration tasks (witness patrols, health checks) -- Polecat work sessions (transient by nature) -- Patrol loops (continuous monitoring) -- Routine operations (no audit value) - -**Rule of thumb**: If you'd regret losing the execution trace, use Mol. If the work is routine and only the outcome matters, use Wisp. - -### Molecule Format - -Molecules are stored as JSONL in `molecules.jsonl`: - -```json -{ - "id": "mol-shiny", - "title": "Shiny: {{feature}}", - "description": "Full workflow from design to merge.", - "labels": ["template"], - "issue_type": "epic" -} -``` - -Steps are hierarchical children with dependencies encoded as beads edges: - -``` -mol-shiny -├── .1 design "Think about architecture" -├── .2 implement "Write the code" ← depends on .1 -├── .3 review "Self-review changes" ← depends on .2 -├── .4 test "Write and run tests" ← depends on .2 -└── .5 submit "Submit for merge" ← depends on .3, .4 -``` - -**Key points:** -- Storage is JSONL (standard beads issue format) -- `{{var}}` placeholders resolved at bond time -- Steps are child issues, not embedded markdown -- Dependencies are beads edges, not text directives - -### Molecule Composition - -Molecules compose via bonding - attaching one molecule to another: - -``` -mol-gastown-polecat (composed) -├── [mol-shiny steps bonded here] -│ ├── .1 design -│ ├── .2 implement -│ ├── .3 review -│ ├── .4 test -│ └── .5 submit -└── .6 install-binary ← depends on .5 (submit) -``` - -**Composition via bond:** -```bash -bd mol bond mol-shiny $PARENT_MOL -bd create "Install binary" --parent=$PARENT_MOL --deps=.5 -``` - -**Semantics:** -- Bonding attaches one molecule's steps as children -- New steps can depend on bonded steps -- Multiple bonds create complex polymers -- Dependencies are standard beads edges - -### Nondeterministic Idempotence - -This is the key property enabling distributed molecule execution: - -1. **Deterministic Structure**: Molecule defines exactly what steps exist and their dependencies -2. **Nondeterministic Execution**: Any worker can execute any ready step -3. **Idempotent Progress**: Completed steps stay completed; re-entry is safe - -**How it works:** - -``` -Worker A picks up "design" (pending → in_progress) -Worker A completes "design" (in_progress → completed) -Worker A dies before "implement" -Worker B queries bd ready, sees "implement" is now ready -Worker B picks up "implement" (any worker can continue) -``` - -This is like a **distributed work queue** backed by beads: -- Beads is the queue (steps are issues with status) -- Git is the persistence layer -- No separate message broker needed -- Full auditability of who did what, when - -### Wisp Molecules: Transient Execution Traces - -**Wisps** are transient execution traces - the "steam" in Gas Town's engine metaphor. Claude is fire; Claude Code is a Steam engine; Gas Town is a Steam Train, with Beads as the tracks. Wisps are steam vapors that dissipate after the work is done. - -**Why wisps?** -- **Observability**: See what's happening during execution without cluttering the permanent ledger -- **Recovery**: Wisps provide checkpoints for crash recovery -- **Compression**: Squash wisps into a digest when done - keep the outcome, discard the trace -- **Clean ledger**: Permanent beads show what was accomplished; wisps show how (temporarily) - -**Wisp workflow:** - -``` -Proto Molecule (template) - │ - ▼ bond - Wisp Molecule (execution) - │ - ┌────┴────┐ - ▼ ▼ - burn squash -(discard) (digest) -``` - -1. **Bond**: Instantiate a proto molecule as a wisp molecule with transient children -2. **Execute**: Agents work on wisp children, marking steps complete -3. **Complete**: When done, either: - - **Burn**: Discard wisps entirely (routine work, no audit needed) - - **Squash**: Compress wisps into a digest (preserve summary of what was done) - -**Wisp structure:** - -``` -gt-abc123 (Proto: shiny) - │ - ▼ bond -gt-abc123.exec-001 (Wisp Molecule) ← wisp=true, parent=gt-abc123 - ├── gt-abc123.exec-001.design ← wisp child - ├── gt-abc123.exec-001.implement ← wisp child - ├── gt-abc123.exec-001.review ← wisp child - └── gt-abc123.exec-001.test ← wisp child - │ - ▼ squash -gt-abc123.digest-001 (Digest) ← wisp=false, permanent -``` - -**Wisp commands:** - -```bash -bd mol bond gt-abc123 # Create wisp molecule from proto -bd mol bond gt-abc123 --template=quick # Use specific template -bd mol squash gt-abc123.exec-001 # Compress wisps to digest -bd mol squash --summary="Agent summary" # With agent-generated summary -bd mol burn gt-abc123.exec-001 # Discard wisps without digest -``` - -**Wisp storage:** - -Wisps are stored in a per-rig wisp database: -- `/.beads-wisp/` - Separate from permanent beads, **gitignored** -- Fast writes, no sync overhead -- Auto-cleaned on squash/burn -- Digests write to permanent beads - -**See**: [wisp-architecture.md](wisp-architecture.md) for full specification including: -- Canonical storage locations -- Role assignments (which agents use wisps) -- Patrol pattern implementation -- Beads implementation requirements - -**Patrols use wisps:** - -Patrol agents (long-running monitors) execute in infinite wisp loops: -1. Execute molecule as wisp -2. Squash to digest (compressed activity record) -3. Sleep/wait for trigger -4. Repeat - -This gives patrols full audit trails without ledger bloat. - -### Dynamic Bonding: The Christmas Ornament Pattern - -Some workflows need **dynamic structure** - steps that emerge at runtime based -on discovered work. Consider mol-witness-patrol: it monitors N polecats where -N varies. A static molecule can't express "for each polecat, do these steps." - -The solution is **dynamic bonding** - creating child molecules at runtime: - -```bash -# In survey-workers step: -for polecat in $(gt polecat list gastown); do - bd mol bond mol-polecat-arm $PATROL_WISP_ID --var polecat_name=$polecat -done -``` - -This creates the **Christmas Ornament** shape: - -``` - ★ mol-witness-patrol (trunk) - /|\ - ┌─────┘ │ └─────┐ - ● ● ● mol-polecat-arm (dynamic arms) - ace nux toast - │ │ │ - ┌──┴──┐ ┌──┴──┐ ┌──┴──┐ - │steps│ │steps│ │steps│ (each arm has 4-5 steps) - └──┬──┘ └──┬──┘ └──┬──┘ - └───────┴───────┘ - │ - ⬣ base (cleanup) -``` - -Arms execute in **parallel**. The `aggregate` step uses `WaitsFor: all-children` -to gate until all dynamically-bonded children complete. - -See [molecular-chemistry.md](molecular-chemistry.md#dynamic-bonding-the-christmas-ornament-pattern) -for full documentation. - -### The Activity Feed - -Dynamic bonding enables a **real-time activity feed** - structured work state -instead of parsing agent logs: - -``` -[14:32:01] ✓ patrol-x7k.inbox-check completed -[14:32:03] ✓ patrol-x7k.check-refinery completed -[14:32:08] + patrol-x7k.arm-ace bonded (5 steps) -[14:32:08] + patrol-x7k.arm-nux bonded (5 steps) -[14:32:09] → patrol-x7k.arm-ace.capture in_progress -[14:32:10] ✓ patrol-x7k.arm-ace.capture completed -[14:32:14] ✓ patrol-x7k.arm-ace.decide completed (action: nudge-1) -[14:32:17] ✓ patrol-x7k.arm-ace COMPLETE -[14:32:23] ✓ patrol-x7k SQUASHED → digest-x7k -``` - -**This is what you want to see.** Not Claude's internal monologue. WORK STATE. - -The beads ledger becomes a real-time activity feed: -- `bd activity --follow` - Stream state transitions -- `bd activity --mol ` - Activity for specific molecule -- Control plane IS data plane - state transitions are queryable data - -This transforms debugging from "read the logs" to "watch the feed." - -### Step States - -``` -pending → in_progress → completed - ↘ failed -``` - -| State | Meaning | -|-------|---------| -| `pending` (open) | Step not yet started, waiting for dependencies | -| `in_progress` | Worker has claimed this step | -| `completed` (closed) | Step finished successfully | -| `failed` | Step failed (needs intervention) | - -**Recovery mechanism:** -- If worker dies mid-step, step stays `in_progress` -- After timeout (default 30 min), step can be reclaimed -- `bd release ` manually releases stuck steps -- Another worker can then pick it up - -### Instantiation - -When a molecule is attached to an issue: - -```bash -gt sling gt-xyz --molecule mol-shiny -``` - -1. Molecule is validated (steps, dependencies) -2. Child beads are created for each step: - - `gt-xyz.design`, `gt-xyz.implement`, etc. -3. Inter-step dependencies are wired -4. First ready step(s) become available via `bd ready` -5. Polecat starts on first ready step - -**Provenance tracking:** -- Each instance has an `instantiated_from` edge to the source molecule -- Enables querying: "show all instances of mol-shiny" - -### Standard Molecules - -Gas Town includes formula files (`.beads/formulas/`) for standard molecules: - -**mol-shiny** (5 steps): -``` -design → implement → review → test → submit -``` -Full quality workflow with design phase and self-review. - -**mol-quick-fix** (3 steps): -``` -implement → test → submit -``` -Fast path for small, well-understood changes. - -**mol-research** (2 steps): -``` -investigate → document -``` -Exploration workflow for understanding problems. - -Formula files support both TOML (preferred) and JSON formats: -```bash -bd formula list # List available formulas -bd formula show shiny # Show formula details -bd formula convert --all # Convert JSON to TOML -bd cook # Cook formula into proto bead -``` - -### Usage - -```bash -# List available molecules (cooked protos) -bd mol list - -# Show molecule details -bd mol show mol-shiny - -# Bond (instantiate) a molecule -bd mol bond mol-shiny --var feature_name="auth" - -# Spawn polecat with molecule -gt sling gt-xyz --molecule mol-shiny -``` - -### Why Molecules? - -**Goal: Nondeterministic Idempotence** - -The idea is that workflows can survive crashes, context compaction, and restarts because state is stored externally in beads, not in agent memory. - -1. **Crash recovery**: Agent dies mid-workflow? Restart and continue from last completed step. No work is lost. -2. **Context survival**: Claude's context compacts? Agent re-reads molecule state from beads and knows exactly where it was. -3. **Quality gates**: Every polecat follows the same review/test workflow, enforced by molecule structure. -4. **Error isolation**: Each step is a checkpoint; failures are contained, not cascading. -5. **Parallelism**: Independent steps can run in parallel across workers. -6. **Auditability**: Full history of who did what step, when - queryable in beads. -7. **Composability**: Build complex workflows from simple building blocks. -8. **Resumability**: Any worker can continue where another left off. - -**Without molecules**: Agents are prompted with instructions, work from memory, and lose state on restart. Autonomous operation is impossible. - -**With molecules**: Agents follow persistent TODO lists that survive anything. Work completion is guaranteed. - -### Molecule vs Template - -Beads has two related concepts: -- **bd template**: User-facing workflow templates with variable substitution -- **gt molecule**: Agent-focused execution templates with step dependencies - -Both use similar structures but different semantics: -- Templates focus on parameterization (`{{variable}}` substitution) -- Molecules focus on execution (step states, nondeterministic dispatch) - -### Config vs Molecule: When to Use Which - -**The Key Principle: If it requires cognition, it's a molecule.** - -| Use Case | Config | Molecule | -|----------|--------|----------| -| Static policy (max workers, timeouts) | ✅ | ❌ | -| Agent workflow (design → implement → test) | ❌ | ✅ | -| Outpost routing preferences | ✅ | ❌ | -| Error recovery with decisions | ❌ | ✅ | -| Environment variables | ✅ | ❌ | -| Multi-step processes that can fail | ❌ | ✅ | - -**Config** (`config.json`, `outposts.yaml`): -- Declarative settings -- No decision-making required -- Read once at startup -- Changes require restart - -**Molecules**: -- Procedural workflows -- Require agent cognition at each step -- Survive agent restarts -- Track progress through step states - -**Example: Outpost Assignment** - -Static policy in `outposts.yaml`: -```yaml -policy: - default_preference: [local, gce-burst, cloudrun-burst] -``` - -But if assignment requires cognition (analyzing work characteristics, checking outpost health, making tradeoffs), escalate to a molecule like `mol-outpost-assign`. - -**The insight**: Gas Town doesn't spawn workers on issues. It spawns workers on molecules. The issue is just the seed data for the molecule execution. - -### Operational Molecules - -Molecules aren't just for implementing features. Any multi-step process that requires cognition, can fail partway through, or needs to survive agent restarts should be a molecule. - -**The key insight**: By encoding operational workflows as molecules, Gas Town gains **nondeterministic idempotence** for system operations, not just work. An agent can crash mid-startup, restart, read its molecule state, and continue from the last completed step. - -#### mol-polecat-work - -The full polecat lifecycle: - -``` -mol-polecat-work -├── .1 load-context "gt prime, bd prime, check inbox" -├── .2 implement "Write code, file discovered work" ← .1 -├── .3 self-review "Check for bugs, security issues" ← .2 -├── .4 verify-tests "Run tests, add coverage" ← .2 -├── .5 rebase-main "Rebase, resolve conflicts" ← .3, .4 -├── .6 submit-merge "Submit to queue, verify CI" ← .5 -├── .7 generate-summary "Squash summary, file remaining work" ← .6 -└── .8 request-shutdown "Signal Witness, wait for kill" ← .7 -``` - -**Why this matters**: A polecat that crashes after step 4 doesn't lose work. On restart, it reads molecule state, sees ".4: completed, .5: pending", and continues rebasing. - -#### mol-rig-activate - -Activating a rig for work: - -``` -mol-rig-activate -├── .1 verify-rig "Check config, git remote" -├── .2 start-witness "Start if needed, verify health" ← .1 -├── .3 start-refinery "Start if needed, verify health" ← .1 -├── .4 sync-beads "Sync from remote" ← .2 -├── .5 identify-ready "Query bd ready, prioritize" ← .4 -└── .6 spawn-workers "Spawn polecats for work" ← .5, .3 -``` - -#### mol-graceful-shutdown - -Shutting down Gas Town properly: - -``` -mol-graceful-shutdown -├── .1 notify-agents "Send shutdown to all agents" -├── .2 wait-squash "Wait for molecule completion" ← .1 -├── .3 verify-clean "Check git states clean" ← .2 -├── .4 kill-workers "Terminate polecats, worktrees" ← .3 -├── .5 kill-core "Terminate Witness, Refinery" ← .4 -└── .6 final-sync "Final beads sync" ← .5 -``` - -**Key principle**: If a multi-step process requires cognition and can fail partway through, it should be a molecule. This applies to: -- Agent lifecycle (startup, work, shutdown) -- System operations (activation, deactivation) -- Batch processing (parallel worker coordination) -- Recovery procedures (doctor --fix) - -### Pluggable Molecules - -Some workflows benefit from **pluggable steps** - dimensions that can be added or removed dynamically. The canonical example is **code review**, where each review dimension (security, performance, test coverage) is a plugin molecule. - -#### Philosophy: Plugins ARE Molecules - -In Gas Town, plugins are molecules with specific labels. No separate format, no YAML configs, no directory conventions. Just beads. - -```json -{ - "id": "mol-security-scan", - "title": "Security scan: {{scope}}", - "description": "Check for OWASP Top 10 vulnerabilities in {{scope}}", - "labels": ["template", "plugin", "code-review", "phase:tactical", "tier:sonnet"], - "issue_type": "task" -} -``` - -**Add a dimension**: Install a plugin molecule (`bd mol install mol-security-scan`) -**Remove a dimension**: Uninstall or don't bond it -**Customize a dimension**: Fork and modify the molecule - -#### Plugin Labels - -Labels encode metadata that would otherwise go in config files: - -| Label | Purpose | -|-------|---------| -| `plugin` | Marks as bondable at hook points | -| `code-review` | Which molecule can use this plugin | -| `phase:tactical` | Grouping for ordering | -| `tier:sonnet` | Model hint | -| `witness` / `deacon` | Which patrol can use it | - -#### Dynamic Assembly - -At runtime, the parent molecule bonds plugins based on labels: - -```bash -# In mol-code-review, during plugin-run step: -for plugin in $(bd mol list --label plugin,code-review --json | jq -r '.[].id'); do - bd mol bond $plugin $CURRENT_MOL --var scope="$SCOPE" -done -``` - -Creates a structure like: -``` -gt-xyz (mol-code-review instance) -├── discovery plugins (parallel) -│ ├── .file-census -│ ├── .dep-graph -│ └── .coverage-map -├── structural plugins (sequential) ← depends on discovery -│ ├── .architecture-review -│ └── .abstraction-analysis -├── tactical plugins (parallel) ← depends on structural -│ ├── .security-scan -│ └── .performance-review -└── .synthesis ← depends on tactical -``` - -#### Code Review Molecule - -The **mol-code-review** molecule is the reference implementation: - -**Discovery Phase** (parallel scouts): -- `mol-file-census` - Inventory: sizes, ages, churn rates -- `mol-dep-graph` - Dependencies, cycles, inversions -- `mol-coverage-map` - Test coverage, dead code - -**Structural Phase** (sequential): -- `mol-architecture-review` - Does structure match domain? -- `mol-abstraction-analysis` - Wrangling at wrong layers? - -**Tactical Phase** (parallel per hotspot): -- `security-scan` - OWASP Top 10, injection, auth bypass -- `performance-review` - N+1 queries, missing caching -- `complexity-analysis` - Cyclomatic > 10, deep nesting -- `test-gaps` - Untested branches, missing edge cases -- `elegance-review` - Magic numbers, unclear names - -**Synthesis Phase** (single coordinator): -- Deduplicate findings -- Establish dependencies between fix-beads -- Prioritize by impact -- Sequence recommendations - -#### Findings Become Beads - -Each review step generates findings as beads: - -``` -gt-sec-001 SQL injection in login() discovered-from: gt-xyz.tactical-security -gt-sec-002 Missing CSRF token discovered-from: gt-xyz.tactical-security -gt-perf-001 N+1 query in dashboard discovered-from: gt-xyz.tactical-performance -``` - -These are the work that "feeds the beast" - the review molecule generates fix beads. - -#### Iteration Without Built-In Loops - -You don't need convergence built into the molecule. Just run it again: - -1. Run `gt molecule instantiate code-review` -2. Workers close all review beads, generate fix beads -3. Fix beads get closed -4. Run `gt molecule instantiate code-review` again -5. Fewer findings this time -6. Repeat until noise floor - -Each instantiation is independent. The ledger shows all runs, enabling comparison. - -#### Static vs Pluggable - -| Aspect | Static Molecule | Pluggable Molecule | -|--------|-----------------|-------------------| -| Definition | Steps defined at template time | Steps bonded at runtime | -| Add step | Edit molecule template | Install plugin molecule | -| Remove step | Edit molecule template | Don't bond it | -| Customization | Edit description | Fork plugin molecule | -| Use case | Fixed workflows | Extensible workflows | - -Both patterns are valid. Use static molecules for well-defined workflows (mol-shiny, mol-polecat-work). Use pluggable molecules when dimensions should be customizable (mol-code-review, mol-witness-patrol). - -## Directory Structure - -### HQ Level - -The HQ (town root) is created by `gt install`: - -``` -~/gt/ # HQ ROOT (Gas Town installation) -├── CLAUDE.md # Mayor role context (runs from here) -├── .beads/ # Town-level beads (prefix: hq-) -│ ├── beads.db # Mayor mail, coordination, handoffs -│ └── config.yaml -│ -├── .runtime/ # Town runtime state (gitignored) -│ ├── daemon.json # Daemon PID, heartbeat -│ ├── deacon.json # Deacon cycle state -│ └── agent-requests.json # Lifecycle requests -│ -├── mayor/ # Mayor configuration and state -│ ├── town.json # {"type": "town", "name": "..."} -│ ├── rigs.json # Registry of managed rigs -│ ├── config.json # Town-level config (theme defaults, etc.) -│ └── state.json # Mayor agent state -│ -├── rigs/ # Standard location for rigs -│ ├── gastown/ # A rig (project container) -│ └── wyvern/ # Another rig -│ -└── / # OR rigs at HQ root (legacy) -``` - -**Notes**: -- Mayor's mail is in town beads (`hq-*` issues), not JSONL files -- Rigs can be in `rigs/` or at HQ root (both work) -- `.runtime/` is gitignored - contains ephemeral process state -- See [docs/hq.md](hq.md) for advanced HQ configurations - -### Rig Level - -Created by `gt rig add `: - -``` -gastown/ # Rig = container (NOT a git clone) -├── config.json # Rig identity only (type, name, git_url, beads.prefix) -├── .beads/ → mayor/rig/.beads # Symlink to canonical beads in Mayor -│ -├── .repo.git/ # BARE REPO: Single git source of truth -│ # All worktrees share this .git database -│ # No working directory (invisible base) -│ -├── settings/ # Rig behavioral config (git-tracked) -│ ├── config.json # Theme, merge_queue, max_workers -│ └── namepool.json # Pool settings (style, max) -│ -├── .runtime/ # Rig runtime state (gitignored) -│ ├── witness.json # Witness process state -│ ├── refinery.json # Refinery process state -│ └── namepool-state.json # In-use names, overflow counter -│ -├── mayor/ # Mayor's per-rig presence -│ ├── rig/ # Worktree (human workspace) -│ │ └── .beads/ # Canonical rig beads (prefix: gt-, etc.) -│ └── state.json -│ -├── refinery/ # Refinery agent (merge queue processor) -│ ├── rig/ # Worktree ON MAIN (can see polecat branches) -│ └── state.json -│ -├── witness/ # Witness agent (per-rig pit boss) -│ └── state.json # No clone needed (monitors polecats) -│ -├── crew/ # Overseer's personal workspaces -│ └── / # Worktree or full clone (TBD) -│ -└── polecats/ # Worker directories (git worktrees) - ├── Nux/ # Worktree on polecat/Nux branch - └── Toast/ # Worktree on polecat/Toast branch -``` - -**Configuration tiers:** -- **Identity** (`config.json`): Rig name, git_url, beads prefix - rarely changes -- **Settings** (`settings/`): Behavioral config - git-tracked, shareable -- **Runtime** (`.runtime/`): Process state - gitignored, transient - -**Git architecture (bare repo pattern):** -- `.repo.git/` is a **bare clone** (no working directory) - shared git database -- Mayor is a **regular clone** (separate `.git/`) - doesn't need branch visibility -- Refinery and polecats are **worktrees** of `.repo.git` - share refs -- Polecat branches are visible to refinery because they share the same .git database -- This is the standard git server pattern - not a hack - -**Why bare repo?** -- No working directory to accidentally work in (invisible base) -- Refinery can see polecat branches immediately (shared refs) -- Only `main` gets pushed to origin (clean remote) -- Faster polecat spawn (worktrees are instant) - -**Beads architecture:** -- Mayor's clone holds the canonical `.beads/` for the rig -- Rig root symlinks `.beads/` → `mayor/rig/.beads` -- All agents (crew, polecats, refinery) inherit beads via parent lookup -- Polecats use redirect files pointing to mayor/rig/.beads - -**Key points:** -- The rig root has no `.git/` - it's a container, not a repository -- `.repo.git/` holds git data for refinery and polecats (worktrees) -- Mayor is a separate clone (doesn't need to see polecat branches) -- Refinery worktree is on `main` for merge operations -- Witness doesn't need a worktree (just monitors polecat state) - -```mermaid -graph TB - subgraph Rig["Rig: gastown (container, NOT a git clone)"] - Config["config.json"] - Beads[".beads/"] - - subgraph Polecats["polecats/"] - Nux["Nux/
(worktree)"] - Toast["Toast/
(worktree)"] - end - - subgraph Refinery["refinery/"] - RefRig["rig/
(canonical main)"] - RefState["state.json"] - end - - subgraph Witness["witness/"] - WitState["state.json"] - end - - subgraph MayorRig["mayor/"] - MayRig["rig/
(git clone)"] - MayState["state.json"] - end - - subgraph Crew["crew/"] - CrewMain["main/
(git clone)"] - end - end - - Beads -.->|BEADS_DIR| Nux - Beads -.->|BEADS_DIR| Toast - Beads -.->|BEADS_DIR| RefRig - Beads -.->|BEADS_DIR| MayRig - Beads -.->|BEADS_DIR| CrewMain -``` - -### ASCII Directory Layout - -For reference without mermaid rendering (see [hq.md](hq.md) for creation/setup): - -``` -~/gt/ # HQ ROOT (Gas Town installation) -├── CLAUDE.md # Mayor role context -├── .beads/ # Town-level beads (gm-* prefix) -│ ├── beads.db # Mayor mail, coordination -│ └── config.yaml -│ -├── mayor/ # Mayor configuration and state -│ ├── town.json # {"type": "town", "name": "..."} -│ ├── rigs.json # Registry of managed rigs -│ └── state.json # Mayor agent state -│ -├── gastown/ # RIG (container, NOT a git clone) -│ ├── config.json # Rig configuration -│ ├── .repo.git/ # Bare repo (shared git database) -│ ├── .beads/ → mayor/rig/.beads # Symlink to Mayor's canonical beads -│ │ -│ ├── mayor/ # Mayor's per-rig presence -│ │ ├── rig/ # Separate clone (canonical beads) -│ │ │ ├── .git/ -│ │ │ ├── .beads/ # CANONICAL rig beads (gt-* prefix) -│ │ │ └── -│ │ └── state.json -│ │ -│ ├── refinery/ # Refinery agent (merge queue) -│ │ ├── rig/ # Worktree of .repo.git (on main) -│ │ │ ├── .git # File pointing to ../.repo.git -│ │ │ └── -│ │ └── state.json -│ │ -│ ├── witness/ # Witness agent (pit boss) -│ │ └── state.json # No clone needed -│ │ -│ ├── crew/ # Overseer's personal workspaces -│ │ └── / # Full clone (inherits beads from rig) -│ │ ├── .git/ -│ │ └── -│ │ -│ ├── polecats/ # Worker directories (worktrees) -│ │ ├── Nux/ # Git worktree from .repo.git -│ │ │ └── # (inherits beads from rig) -│ │ └── Toast/ # Git worktree from .repo.git -│ │ -│ └── molecules.jsonl # Optional local molecule catalog -│ -└── wyvern/ # Another rig (same structure) - ├── config.json - ├── .beads/ → mayor/rig/.beads - ├── mayor/ - ├── refinery/ - ├── witness/ - ├── crew/ - └── polecats/ -``` - -**Key changes from earlier design:** -- Town beads (`gm-*`) hold Mayor mail instead of JSONL files -- Mayor has per-rig clone that's canonical for beads -- `.repo.git/` bare repo serves as source for refinery/polecat worktrees -- Rig `.beads/` symlinks to Mayor's canonical beads -- Polecats are git worktrees from `.repo.git` (fast, share refs) - -### Why Decentralized? - -Agents live IN rigs rather than in a central location: -- **Locality**: Each agent works in the context of its rig -- **Independence**: Rigs can be added/removed without restructuring -- **Parallelism**: Multiple rigs can have active workers simultaneously -- **Simplicity**: Agent finds its context by looking at its own directory - -## Agent Responsibilities - -### Mayor - -The Mayor is the global coordinator: -- **Work dispatch**: Spawns workers for issues, coordinates batch work on epics -- **Cross-rig coordination**: Routes work between rigs when needed -- **Escalation handling**: Resolves issues Witnesses can't handle -- **Strategic decisions**: Architecture, priorities, integration planning - -**NOT Mayor's job**: Per-worker cleanup, session killing, nudging workers - -### Witness - -The Witness is the per-rig "pit boss": -- **Worker monitoring**: Track polecat health and progress -- **Nudging**: Prompt workers toward completion -- **Pre-kill verification**: Ensure git state is clean before killing sessions -- **Session lifecycle**: Kill sessions, update worker state -- **Self-cycling**: Hand off to fresh session when context fills -- **Escalation**: Report stuck workers to Mayor - -**Key principle**: Witness owns ALL per-worker cleanup. Mayor is never involved in routine worker management. - -### Refinery - -The Refinery manages the merge queue: -- **PR review**: Check polecat work before merging -- **Integration**: Merge completed work to main -- **Conflict resolution**: Handle merge conflicts -- **Quality gate**: Ensure tests pass, code quality maintained - -```mermaid -flowchart LR - subgraph "Merge Queue Flow" - P1[Polecat 1
branch] --> Q[Merge Queue] - P2[Polecat 2
branch] --> Q - P3[Polecat 3
branch] --> Q - Q --> R{Refinery} - R -->|merge| M[main] - R -->|conflict| P1 - end -``` - -#### Direct Landing (Bypass Merge Queue) - -Sometimes Mayor needs to land a polecat's work directly, skipping the Refinery: - -| Scenario | Use Direct Landing? | -|----------|---------------------| -| Single polecat, simple change | Yes | -| Urgent hotfix | Yes | -| Refinery unavailable | Yes | -| Multiple polecats, potential conflicts | No - use Refinery | -| Complex changes needing review | No - use Refinery | - -**Commands:** - -```bash -# Normal flow (through Refinery) -gt merge-queue add # Polecat signals PR ready -gt refinery process # Refinery processes queue - -# Direct landing (Mayor bypasses Refinery) -gt land --direct / # Land directly to main -gt land --direct --force / # Skip safety checks -gt land --direct --skip-tests / # Skip test run -gt land --direct --dry-run / # Preview only -``` - -**Direct landing workflow:** - -```mermaid -sequenceDiagram - participant M as 🎩 Mayor - participant R as Refinery Clone - participant P as Polecat Branch - participant B as 📦 Beads - - M->>M: Verify polecat session terminated - M->>P: Check git state clean - M->>R: Fetch polecat branch - M->>R: Merge to main (fast-forward or merge commit) - M->>R: Run tests (optional) - M->>R: Push to origin - M->>B: Close associated issue - M->>P: Delete polecat branch (cleanup) -``` - -**Safety checks (skippable with --force):** -1. Polecat session must be terminated -2. Git working tree must be clean -3. No merge conflicts with main -4. Tests pass (skippable with --skip-tests) - -**When direct landing makes sense:** -- Mayor is doing sequential work without parallel polecats -- Single worker completed an isolated task -- Hotfix needs to land immediately -- Refinery agent is down or unavailable - -### Polecat - -Polecats are the workers that do actual implementation: -- **Molecule execution**: Execute wisp molecules (not just "work on issues") -- **Self-verification**: Run decommission checklist before signaling done -- **Beads access**: Create issues for discovered work, close completed work -- **Clean handoff**: Ensure git state is clean for Witness verification -- **Shutdown request**: Request own termination via `gt handoff` (bottom-up lifecycle) - -**Polecats are like wisps**: They exist only while working. When done, they request shutdown and are deleted (worktree removed, branch deleted). There is no "idle pool" of polecats. - -**Polecat workflow** (molecule-first): -1. Spawn receives issue + proto molecule template -2. Bond creates wisp molecule from proto -3. Polecat executes wisp steps (design → implement → test → submit) -4. On completion, polecat generates summary and squashes wisps to digest -5. Request shutdown, get deleted - -The polecat itself is transient, and so is its execution trace (wisps). Only the digest survives. - -## Key Workflows - -### Work Dispatch - -Work flows through the system as a stream. The Overseer spawns workers, they process issues, and completed work enters the merge queue. - -```mermaid -sequenceDiagram - participant O as 👤 Overseer - participant M as 🎩 Mayor - participant W as 👁 Witness - participant P as 🐱 Polecats - participant R as 🔧 Refinery - - O->>M: Spawn workers for epic - M->>W: Assign issues to workers - W->>P: Start work - - loop For each worker - P->>P: Work on issue - P->>R: Submit to merge queue - R->>R: Review & merge - end - - R->>M: All work merged - M->>O: Report results -``` - -**Note**: There is no "swarm ID" or batch boundary. Workers process issues independently. The merge queue handles coordination. Tackling an epic with parallel polecats is just spawning multiple workers for the epic's child issues. - -### Worker Cleanup (Witness-Owned) - -```mermaid -sequenceDiagram - participant P as 🐱 Polecat - participant W as 👁 Witness - participant M as 🎩 Mayor - participant O as 👤 Overseer - - P->>P: Complete work - P->>W: Done signal - - W->>W: Capture git state - W->>W: Assess cleanliness - - alt Git state dirty - W->>P: Nudge (fix issues) - P->>P: Fix issues - P->>W: Done signal (retry) - end - - alt Clean after ≤3 tries - W->>W: Verify clean - W->>P: Kill session - else Stuck after 3 tries - W->>M: Escalate - alt Mayor can fix - M->>W: Resolution - else Mayor can't fix - M->>O: Escalate to human - O->>M: Decision - end - end -``` - -### Polecat Shutdown Protocol (Bottom-Up) - -Polecats initiate their own shutdown. This enables streaming - workers come and go continuously without artificial batch boundaries. - -```mermaid -sequenceDiagram - participant P as 🐱 Polecat - participant R as 🔧 Refinery - participant W as 👁 Witness - participant B as 📦 Beads - - P->>P: Complete wisp steps - P->>P: Generate summary - P->>B: Squash wisps → digest - P->>R: Submit to merge queue - P->>P: Run gt handoff - - Note over P: Verify git clean,
PR exists,
wisps squashed - - P->>W: Mail: "Shutdown request" - P->>P: Set state = pending_shutdown - - W->>W: Verify safe to kill - W->>P: Kill session - W->>W: git worktree remove - W->>W: git branch -d -``` - -**Key change**: Polecats generate their own summaries and squash wisps before handoff. The digest is the permanent record of what the polecat accomplished. This keeps beads as a pure tool - agents provide the intelligence for summarization. - -**gt handoff command** (run by polecat): -1. Verify git state clean (no uncommitted changes) -2. Verify work handed off (PR created or in queue) -3. Send mail to Witness requesting shutdown -4. Wait for Witness to kill session (don't self-exit) - -**Witness shutdown handler**: -1. Receive shutdown request -2. Verify PR merged or queued, no data loss risk -3. Kill session: `gt session stop /` -4. Remove worktree: `git worktree remove polecats/` -5. Delete branch: `git branch -d polecat/` - -**Why bottom-up?** In streaming, there's no "swarm end" to trigger cleanup. Each worker manages its own lifecycle. The Witness is the lifecycle authority that executes the actual termination. - -### Session Cycling (Mail-to-Self) - -When an agent's context fills, it hands off to its next session: - -1. **Recognize**: Notice context filling (slow responses, losing track of state) -2. **Capture**: Gather current state (active work, pending decisions, warnings) -3. **Compose**: Write structured handoff note -4. **Send**: Mail handoff to own inbox -5. **Exit**: End session cleanly -6. **Resume**: New session reads handoff, picks up where old session left off - -```mermaid -sequenceDiagram - participant S1 as Agent Session 1 - participant MB as 📬 Mailbox - participant S2 as Agent Session 2 - - S1->>S1: Context filling up - S1->>S1: Capture current state - S1->>MB: Send handoff note - S1->>S1: Exit cleanly - - Note over S1,S2: Session boundary - - S2->>MB: Check inbox - MB->>S2: Handoff note - S2->>S2: Resume from handoff state -``` - -## Key Design Decisions - -### 1. Witness Owns Worker Cleanup - -**Decision**: Witness handles all per-worker cleanup. Mayor is never involved. - -**Rationale**: -- Separation of concerns (Mayor strategic, Witness operational) -- Reduced coordination overhead -- Faster shutdown -- Cleaner escalation path - -### 2. Polecats Have Direct Beads Access - -**Decision**: Polecats can create, update, and close beads issues directly. - -**Rationale**: -- Simplifies architecture (no proxy through Witness) -- Empowers workers to file discovered work -- Faster feedback loop -- Beads v0.30.0+ handles multi-agent conflicts - -### 3. Session Cycling via Mail-to-Self - -**Decision**: Agents mail handoff notes to themselves when cycling sessions. - -**Rationale**: -- Consistent pattern across all agent types -- Timestamped and logged -- Works with existing inbox infrastructure -- Clean separation between sessions - -### 4. Decentralized Agent Architecture - -**Decision**: Agents live in rigs (`/witness/rig/`) not centralized (`mayor/rigs//`). - -**Rationale**: -- Agents work in context of their rig -- Rigs are independent units -- Simpler role detection -- Cleaner directory structure - -### 5. Three-Tier Configuration Architecture - -**Decision**: Separate identity, settings, and runtime into distinct locations: -- **Identity** (`config.json`): Rig name, git_url, beads prefix - rarely changes -- **Settings** (`settings/`): Behavioral config - git-tracked, shareable, visible -- **Runtime** (`.runtime/`): Process state - gitignored, transient - -**Rationale**: -- AI models often miss hidden directories (so `.gastown/` was bad) -- Identity config rarely changes; behavioral config may change often -- Runtime state should never be committed -- `settings/` is visible to agents, unlike hidden `.gastown/` - -### 6. Rig as Container, Not Clone - -**Decision**: The rig directory is a pure container, not a git clone of the project. - -**Rationale**: -- **Prevents confusion**: Agents historically get lost (polecats in refinery, mayor in polecat dirs). If the rig root were a clone, it's another tempting target for confused agents. Two confused agents at once = collision disaster. -- **Single work location**: Each agent has exactly one place to work (their own `/rig/` clone) -- **Clear role detection**: "Am I in a `/rig/` directory?" = I'm in an agent clone -- **Refinery is canonical main**: Refinery's clone serves as the authoritative "main branch" - it pulls, merges PRs, and pushes. No need for a separate rig-root clone. - -### 7. Plugins as Agents - -**Decision**: Plugins are just additional agents with identities, mailboxes, and access to beads. No special plugin infrastructure. - -**Rationale**: -- Fits Gas Town's intentionally rough aesthetic -- Zero new infrastructure needed (uses existing mail, beads, identities) -- Composable - plugins can invoke other plugins via mail -- Debuggable - just look at mail logs and bead history -- Extensible - anyone can add a plugin molecule to their catalog - -**Structure**: Plugin molecules in `molecules.jsonl` with labels for discovery. - -### 8. Rig-Level Beads via BEADS_DIR - -**Decision**: Each rig has its own `.beads/` directory. Agents use the `BEADS_DIR` environment variable to point to it. - -**Rationale**: -- **Centralized issue tracking**: All polecats in a rig share the same beads database -- **Project separation**: Even if the project repo has its own `.beads/`, Gas Town agents use the rig's beads instead -- **OSS-friendly**: For contributing to projects you don't own, rig beads stay separate from upstream -- **Already supported**: Beads supports `BEADS_DIR` env var (see beads `internal/beads/beads.go`) - -**Configuration**: Gas Town sets `BEADS_DIR` when spawning agents: -```bash -export BEADS_DIR=/path/to/rig/.beads -``` - -**See also**: beads issue `bd-411u` for documentation of this pattern. - -### 9. Direct Landing Option - -**Decision**: Mayor can land polecat work directly, bypassing the Refinery merge queue. - -**Rationale**: -- **Flexibility**: Not all work needs merge queue overhead -- **Sequential work**: Mayor doing single-polecat work shouldn't need Refinery -- **Emergency path**: Hotfixes can land immediately -- **Resilience**: System works even if Refinery is down - -**Constraints**: -- Direct landing still uses Refinery's clone as the canonical main -- Safety checks prevent landing dirty or conflicting work -- Mayor takes responsibility for quality (no Refinery review) - -**Commands**: -```bash -gt land --direct / # Standard direct land -gt land --direct --force / # Skip safety checks -``` - -### 10. Beads Daemon Awareness - -**Decision**: Gas Town must disable the beads daemon for worktree-based polecats. - -**Rationale**: -- The beads daemon doesn't track which branch each worktree has checked out -- Daemon can commit beads changes to the wrong branch -- This is a beads limitation, not a Gas Town bug -- Full clones don't have this problem - -**Configuration**: -```bash -# For worktree polecats (REQUIRED) -export BEADS_NO_DAEMON=1 - -# For full-clone polecats (optional) -# Daemon is safe, no special config needed -``` - -**See also**: beads docs/WORKTREES.md and docs/DAEMON.md for details. - -### 11. Work is a Stream (No Swarm IDs) - -**Decision**: Work state is encoded in beads epics and issues. There are no "swarm IDs" or separate swarm infrastructure - the epic IS the grouping, the merge queue IS the coordination. - -**Rationale**: -- **No new infrastructure**: Beads already provides hierarchy, dependencies, status, priority -- **Shared state**: All rig agents share the same `.beads/` via BEADS_DIR -- **Queryable**: `bd ready` finds work with no blockers, enabling multi-wave orchestration -- **Auditable**: Beads history shows work progression -- **Resilient**: Beads sync handles multi-agent conflicts -- **No boundary problem**: When does a swarm start/end? Who's in it? These questions dissolve - work is a stream - -**How it works**: -- Create an epic with child issues for batch work -- Dependencies encode ordering (task B depends on task A) -- Status transitions track progress (open → in_progress → closed) -- Witness queries `bd ready` to find next available work -- Spawn workers as needed - add more anytime -- Batch complete = all child issues closed (or just keep going) - -**Example**: Batch work on authentication bugs: -``` -gt-auth-epic # Epic: "Fix authentication bugs" -├── gt-auth-epic.1 # "Fix login timeout" (ready, no deps) -├── gt-auth-epic.2 # "Fix session expiry" (ready, no deps) -└── gt-auth-epic.3 # "Update auth tests" (blocked by .1 and .2) -``` - -Workers process issues independently. Work flows through the merge queue. No "swarm ID" needed - the epic provides grouping, labels provide ad-hoc queries, dependencies provide sequencing. - -### 12. Agent Session Lifecycle (One Daemon) - -**Decision**: ONE daemon (Go process) for all Gas Town manages agent lifecycles. Agents use a unified `gt handoff` command to request lifecycle actions. - -**Architecture**: -``` -Gas Town Daemon (gt daemon) -├── Pokes Mayor periodically -├── Pokes all Witnesses periodically -├── Processes lifecycle requests from deacon/ inbox -└── Restarts sessions when cycle requested - -Lifecycle Hierarchy: - Daemon → manages Mayor, all Witnesses - Witness → manages Polecats, Refinery (per rig) -``` - -**Rationale**: -- Agents can't restart themselves after exiting -- ONE daemon is simpler than per-rig daemons -- Daemon is dumb scheduler; intelligence is in agents -- Unified protocol means all agents work the same way - -**Unified lifecycle command** (`gt handoff`): -```bash -gt handoff # Context-aware default -gt handoff --shutdown # Terminate, don't restart (polecats) -gt handoff --cycle # Restart with handoff (long-running agents) -gt handoff --restart # Fresh restart, no handoff -``` - -| Agent | Default | Sends request to | -|-------|---------|------------------| -| Polecat | --shutdown | rig/witness | -| Refinery | --cycle | rig/witness | -| Witness | --cycle | deacon/ | -| Mayor | --cycle | deacon/ | - -**Lifecycle request protocol**: -1. Agent runs `gt handoff` (verifies git clean, sends handoff mail) -2. Agent sends lifecycle request to its manager -3. Agent sets `requesting_: true` in state.json -4. Agent waits (does NOT self-exit) -5. Manager receives request, verifies safe -6. Manager kills session -7. Manager starts new session (for cycle/restart) -8. New session reads handoff mail, resumes work - -**Daemon heartbeat loop**: -- Poke Mayor: "HEARTBEAT: check your rigs" -- Poke each Witness: "HEARTBEAT: check your workers" -- Agents ignore poke if already working -- Process any lifecycle requests in deacon/ inbox -- Restart dead sessions if cycle was requested - -```mermaid -sequenceDiagram - participant A1 as Agent Session 1 - participant M as Lifecycle Manager - participant A2 as Agent Session 2 - - A1->>A1: gt handoff --cycle - A1->>A1: Send handoff mail to self - A1->>M: Lifecycle request: cycle - A1->>A1: Set requesting_cycle, wait - - M->>M: Verify safe to act - M->>A1: Kill session - M->>A2: Start new session - A2->>A2: Read handoff mail - A2->>A2: Resume work -``` - -**Polecat shutdown** (--shutdown default): -After Witness kills session: -- Remove worktree: `git worktree remove polecats/` -- Delete branch: `git branch -d polecat/` -- Polecat ceases to exist (transient) - -### 13. Resource-Constrained Worker Pool - -**Decision**: Each rig has a configurable `max_workers` limit for concurrent polecats. - -**Rationale**: -- Claude Code can use 500MB+ RAM per session -- Prevents resource exhaustion on smaller machines -- Enables autonomous operation without human oversight -- Witness respects limit when spawning new workers - -**Configuration** (in rig config.json): -```json -{ - "type": "rig", - "max_workers": 8, - "worker_spawn_delay": "5s" -} -``` - -**Witness behavior**: -- Query active worker count before spawning -- If at limit, wait for workers to complete -- Prioritize higher-priority ready issues - -### 14. Outpost Abstraction for Federation - -**Decision**: Federation uses an "Outpost" abstraction to support multiple compute backends (local, SSH/VM, Cloud Run, etc.) through a unified interface. - -**Rationale**: -- Different workloads need different compute: burst vs long-running, cheap vs fast -- Cloud Run's pay-per-use model is ideal for elastic burst capacity -- VMs are better for autonomous long-running work -- Local is always the default for development -- Platform flexibility lets users choose based on their needs and budget - -**Key insight**: Cloud Run's persistent HTTP/2 connections solve the "zero to one" cold start problem, making container workers viable for interactive-ish work at ~$0.017 per 5-minute session. - -**Design principles**: -1. **Local-first** - Remote outposts are overflow, not primary -2. **Git remains source of truth** - All outposts sync via git -3. **HTTP for Cloud Run** - Don't force filesystem mail onto containers -4. **Graceful degradation** - System works with any subset of outposts - -**See**: `docs/federation-design.md` for full architectural analysis. - -## Multi-Wave Work Processing - -For large task trees (like implementing GGT itself), workers can process multiple "waves" of work automatically based on the dependency graph. - -### Wave Orchestration - -A wave is not explicitly managed - it emerges from dependencies: - -1. **Wave 1**: All issues with no dependencies (`bd ready`) -2. **Wave 2**: Issues whose dependencies are now closed -3. **Wave N**: Continue until all work is done - -```mermaid -graph TD - subgraph "Wave 1 (no dependencies)" - A[Task A] - B[Task B] - C[Task C] - end - - subgraph "Wave 2 (depends on Wave 1)" - D[Task D] - E[Task E] - end - - subgraph "Wave 3 (depends on Wave 2)" - F[Task F] - end - - A --> D - B --> D - C --> E - D --> F - E --> F -``` - -### Witness Work Loop - -``` -while epic has open issues: - ready_issues = bd ready --parent - - if ready_issues is empty and workers_active: - wait for worker completion - continue - - for issue in ready_issues: - if active_workers < max_workers: - spawn worker for issue - else: - break # wait for capacity - - monitor workers, handle completions - -all work complete - report to Mayor -``` - -### Long-Running Autonomy - -With daemon session cycling, the system can run autonomously for extended periods: - -- **Witness cycles**: Every few hours as context fills -- **Refinery cycles**: As merge queue grows complex -- **Workers cycle**: If individual tasks are very large -- **Daemon persistence**: Survives all agent restarts - -The daemon is the only truly persistent component. All agents are transient sessions that hand off state via mail. - -Work is a continuous stream - you can add new issues, spawn new workers, reprioritize the queue, all without "starting a new swarm" or managing batch boundaries. - -## Configuration - -### town.json - -```json -{ - "type": "town", - "version": 1, - "name": "stevey-gastown", - "created_at": "2024-01-15T10:30:00Z" -} -``` - -### rigs.json - -```json -{ - "version": 1, - "rigs": { - "wyvern": { - "git_url": "https://github.com/steveyegge/wyvern", - "added_at": "2024-01-15T10:30:00Z" - } - } -} -``` - -### rig.json (Per-Rig Identity) - -Each rig has a `config.json` at its root containing **identity only** (rarely changes): - -```json -{ - "type": "rig", - "version": 1, - "name": "wyvern", - "git_url": "https://github.com/steveyegge/wyvern", - "beads": { - "prefix": "wyv", - "sync_remote": "origin" // Optional: git remote for bd sync - } -} -``` - -Behavioral settings live in `settings/config.json` (git-tracked, shareable): - -```json -{ - "theme": "desert", - "merge_queue": { - "enabled": true, - "auto_merge": true - }, - "max_workers": 5 -} -``` - -Runtime state lives in `.runtime/` (gitignored, transient): - -```json -// .runtime/witness.json -{ - "state": "running", - "started_at": "2024-01-15T10:30:00Z", - "stats": { "polecats_spawned": 42 } -} -``` - -The rig's `.beads/` directory is always at the rig root. Gas Town: -1. Creates `.beads/` when adding a rig (`gt rig add`) -2. Runs `bd init --prefix ` to initialize it -3. Sets `BEADS_DIR` environment variable when spawning agents - -This ensures all agents in the rig share a single beads database, separate from any beads the project itself might use. - -## CLI Commands - -### HQ Management - -```bash -gt install [path] # Create Gas Town HQ (see hq.md) -gt install --git # Also initialize git with .gitignore -gt install --github=u/r # Also create GitHub repo -gt git-init # Initialize git for existing HQ -gt doctor # Check workspace health -gt doctor --fix # Auto-fix issues -``` - -### Agent Operations - -```bash -gt status # Overall town status -gt rigs # List all rigs -gt polecats # List polecats in a rig -``` - -### Communication - -```bash -gt inbox # Check inbox -gt send -s "Subject" -m "Message" -gt inject "Message" # Direct injection to session -gt capture "" # Run command in polecat session -``` - -### Session Management - -```bash -gt sling --molecule mol-shiny # Spawn polecat with workflow -gt handoff # Polecat requests shutdown (run when done) -gt session stop

# Kill polecat session (Witness uses this) -``` - -**Note**: `gt wake` and `gt sleep` are deprecated - polecats are transient, not pooled. - -### Landing & Merge Queue - -```bash -gt merge-queue add # Add to merge queue (normal flow) -gt merge-queue list # Show pending merges -gt refinery process # Trigger Refinery to process queue - -gt land --direct / # Direct landing (bypass Refinery) -gt land --direct --force ... # Skip safety checks -gt land --direct --skip-tests ... # Skip test verification -gt land --direct --dry-run ... # Preview only -``` - -### Emergency Operations - -```bash -gt stop --all # Kill ALL sessions (emergency halt) -gt stop --rig # Kill all sessions in one rig -gt doctor --fix # Auto-repair common issues -``` - -## Plugins - -Gas Town supports **plugins** - but in the simplest possible way: plugins are just more agents. - -### Philosophy - -Gas Town is intentionally rough and lightweight. A "credible plugin system" with manifests, schemas, and invocation frameworks would be pretentious for a project named after a Mad Max wasteland. Instead, plugins follow the same patterns as all Gas Town agents: - -- **Identity**: Plugins have persistent identities like polecats and witnesses -- **Communication**: Plugins use mail for input/output -- **Artifacts**: Plugins produce beads, files, or other handoff artifacts -- **Lifecycle**: Plugins can be invoked on-demand or at specific workflow points - -### Plugin Structure - -Plugins are molecules with specific labels, stored in `molecules.jsonl`: - -```json -{ - "id": "mol-merge-oracle", - "title": "Analyze merge queue", - "description": "Analyze pending changesets for conflicts and ordering.", - "labels": ["template", "plugin", "refinery", "tier:sonnet"], - "issue_type": "task" -} -``` - -### Invoking Plugins - -Plugins are bonded during patrol execution: - -```bash -# In mol-refinery-patrol plugin-run step: -bd mol bond mol-merge-oracle $PATROL_WISP \ - --var queue_state="$QUEUE_JSON" -``` - -Plugins execute as molecule steps, creating any necessary artifacts (beads, files, branches). - -### Hook Points - -Patrol molecules bond plugins at specific points. Discovery uses labels: - -| Workflow Point | Patrol | Label Filter | -|----------------|--------|--------------| -| Refinery patrol | mol-refinery-patrol | `plugin,refinery` | -| Witness patrol | mol-witness-patrol | `plugin,witness` | -| Deacon patrol | mol-deacon-patrol | `plugin,deacon` | - -Configuration is via labels - agents bond plugins that match their role. - -### Example: Merge Oracle - -The **merge-oracle** plugin analyzes changesets before the Refinery processes them: - -**Input** (via mail from Refinery): -- List of pending changesets -- Current merge queue state - -**Processing**: -1. Build overlap graph (which changesets touch same files/regions) -2. Classify disjointness (fully disjoint → parallel safe, overlapping → needs sequencing) -3. Use LLM to assess semantic complexity of overlapping components -4. Identify high-risk patterns (deletions vs modifications, conflicting business logic) - -**Output**: -- Bead with merge plan (parallel groups, sequential chains) -- Mail to Refinery with recommendation (proceed / escalate to Mayor) -- If escalation needed: mail to Mayor with explanation - -The mol-merge-oracle's description contains the prompts and classification criteria. Gas Town doesn't need to know the internals. - -### Example: Plan Oracle - -The **plan-oracle** plugin helps decompose work: - -**Input**: An issue/epic that needs breakdown - -**Processing**: -1. Analyze the scope and requirements -2. Identify dependencies and blockers -3. Estimate complexity (for parallelization decisions) -4. Suggest task breakdown - -**Output**: -- Beads for the sub-tasks (created via `bd create`) -- Dependency links (via `bd dep add`) -- Mail back with summary and recommendations - -### Example: Beads Hygiene - -The **beads-hygiene** plugin detects and fixes cross-pollution between nested beads databases. - -**Background**: Gas Town has a two-level beads architecture: -- **Town-level** (`~/gt/.beads/`): Mayor mail, cross-rig coordination, HQ-level issues -- **Rig-level** (`~/gt//.beads/`): Project-specific work (bugs, features, tasks) - -Workers sometimes get confused about which database they're in, especially when: -- Their cwd is in a rig but they interact with town-level beads -- They reference issues from the wrong level as dependencies -- They create issues with mismatched prefixes for their context - -**Input** (periodic scan or on-demand via mail): -- List of all beads databases in the town -- Recent issue creation/update activity -- Agent identity and expected context - -**Processing**: -1. Scan each beads database for prefix mismatches - - Town-level should have `hq-*` prefix (headquarters) - - Rig-level should have rig prefix (e.g., `gt-*` for gastown) -2. Check for cross-level dependency references - - Flag `gt-*` issues that depend on `hq-*` (usually wrong) -3. Analyze recent activity for context confusion - - Agent in `gastown/` creating HQ-level issues - - Agent at town level creating rig-specific issues -4. Identify misfiled issues that should be moved - -**Output**: -- Report of detected issues (via mail to Mayor) -- For each misfiled issue: - - Original location and ID - - Suggested correct location - - Confidence level (definite misfile vs. ambiguous) -- Optionally: auto-move with `--fix` flag - -**Hook Points**: -- Witness can invoke before spawning polecats (sanity check) -- Mayor can invoke periodically (nightly hygiene scan) -- Any agent can invoke on-demand when confused - -**mol-beads-hygiene description (prompt core)**: -``` -You are reviewing beads databases for cross-pollution between Gas Town's -two-level architecture: - -TOWN LEVEL (~/gt/.beads/): Coordination, mayor mail, HQ issues - - Prefix: hq-* (headquarters) - - Contains: cross-rig coordination, strategic planning, HQ bugs - -RIG LEVEL (~/gt//.beads/): Project-specific work - - Prefix: -* (e.g., gt-* for gastown rig) - - Contains: bugs, features, tasks for that project - -COMMON MISTAKES TO DETECT: -1. Issue created at wrong level (check prefix vs location) -2. Cross-level dependencies (usually wrong unless intentional) -3. Agent identity mismatch (polecat creating town-level issues) -4. Duplicate issues across levels (same title/description) - -For each issue found, report: -- Issue ID and title -- Current location -- Why it appears misfiled -- Recommended action (move, merge, or leave with note) -``` - -### Why This Design - -1. **Fits Gas Town's aesthetic**: Rough, text-based, agent-shaped -2. **Zero new infrastructure**: Uses existing mail, beads, identities -3. **Composable**: Plugins can invoke other plugins -4. **Debuggable**: Just look at mail logs and bead history -5. **Extensible**: Anyone can add a plugin by creating a directory - -### Plugin Discovery - -```bash -gt plugins # List plugins in a rig -gt plugin status # Check plugin state -``` - -Or just `ls /plugins/`. - -## Failure Modes and Recovery - -Gas Town is designed for resilience. Common failure modes and their recovery: - -| Failure | Detection | Recovery | -|---------|-----------|----------| -| Agent crash | Session gone, state shows 'working' | `gt doctor` detects, reset state to idle | -| Git dirty state | Witness pre-kill check fails | Nudge worker, or manual commit/discard | -| Beads sync conflict | `bd sync` fails | Beads tombstones handle most cases | -| Tmux crash | All sessions inaccessible | `gt doctor --fix` cleans up | -| Stuck work | No progress for 30+ minutes | Witness escalates, Overseer intervenes | -| Disk full | Write operations fail | Clean logs, remove old clones | - -### Recovery Principles - -1. **Fail safe**: Prefer stopping over corrupting data -2. **State is recoverable**: Git and beads have built-in recovery -3. **Doctor heals**: `gt doctor --fix` handles common issues -4. **Emergency stop**: `gt stop --all` as last resort -5. **Human escalation**: Some failures need Overseer intervention - -### Doctor Checks - -`gt doctor` performs health checks at both workspace and rig levels: - -**Workspace checks**: Config validity, Mayor mailbox, rig registry -**Rig checks**: Git state, clone health, Witness/Refinery presence -**Work checks**: Stuck detection, zombie sessions, heartbeat health - -Run `gt doctor` regularly. Run `gt doctor --fix` to auto-repair issues. - -## Federation: Outposts - -Federation enables Gas Town to scale across machines via **Outposts** - remote compute environments that can run workers. - -**Full design**: See `docs/federation-design.md` - -### Outpost Types - -| Type | Description | Cost Model | Best For | -|------|-------------|------------|----------| -| Local | Current tmux model | Free | Development, primary work | -| SSH/VM | Full Gas Town clone on VM | Always-on | Long-running, autonomous | -| CloudRun | Container workers on GCP | Pay-per-use | Burst, elastic, background | - -### Core Abstraction - -```go -type Outpost interface { - Name() string - Type() OutpostType // local, ssh, cloudrun - MaxWorkers() int - ActiveWorkers() int - Spawn(issue string, config WorkerConfig) (Worker, error) - Workers() []Worker - Ping() error -} - -type Worker interface { - ID() string - Outpost() string - Status() WorkerStatus // idle, working, done, failed - Issue() string - Attach() error // for interactive outposts - Logs() (io.Reader, error) - Stop() error -} -``` - -### Configuration - -```yaml -# ~/gt/config/outposts.yaml -outposts: - - name: local - type: local - max_workers: 4 - - - name: gce-burst - type: ssh - host: 10.0.0.5 - user: steve - town_path: /home/steve/ai - max_workers: 8 - - - name: cloudrun-burst - type: cloudrun - project: my-gcp-project - region: us-central1 - service: gastown-worker - max_workers: 20 - cost_cap_hourly: 5.00 - -policy: - default_preference: [local, gce-burst, cloudrun-burst] -``` - -### Cloud Run Workers - -Cloud Run enables elastic, pay-per-use workers: -- **Persistent HTTP/2 connections** solve cold start (zero-to-one) problem -- **Cost**: ~$0.017 per 5-minute worker session -- **Scaling**: 0→N automatically based on demand -- **When idle**: Scales to zero, costs nothing - -Workers receive work via HTTP, clone code from git, run Claude, push results. No filesystem mail needed - HTTP is the control plane. - -### SSH/VM Outposts - -Full Gas Town clone on remote machines: -- **Model**: Complete town installation via SSH -- **Workers**: Remote tmux sessions -- **Sync**: Git for code and beads -- **Good for**: Long-running work, full autonomy if disconnected - -### Design Principles - -1. **Outpost abstraction** - Support multiple backends via unified interface -2. **Local-first** - Remote outposts are for overflow/burst, not primary -3. **Git as source of truth** - Code and beads sync everywhere -4. **HTTP for Cloud Run** - Don't force mail onto stateless containers -5. **Graceful degradation** - System works with any subset of outposts - -### Architecture Diagram - -``` -┌─────────────────────────────────────────────────────────────┐ -│ MAYOR │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ Outpost Manager │ │ -│ │ - Tracks all registered outposts │ │ -│ │ - Routes work to appropriate outpost │ │ -│ │ - Monitors worker status across outposts │ │ -│ └──────────────────────────────────────────────────────┘ │ -│ │ │ │ │ -│ ▼ ▼ ▼ │ -│ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │ -│ │ Local │ │ SSH │ │ CloudRun │ │ -│ │ Outpost │ │ Outpost │ │ Outpost │ │ -│ └────┬─────┘ └────┬─────┘ └──────┬───────┘ │ -└───────┼──────────────┼──────────────────┼───────────────────┘ - │ │ │ - ▼ ▼ ▼ - ┌─────────┐ ┌─────────┐ ┌─────────────┐ - │ tmux │ │ SSH │ │ HTTP/2 │ - │ panes │ │sessions │ │ connections │ - └─────────┘ └─────────┘ └─────────────┘ - │ │ │ - └──────────────┼──────────────────┘ - ▼ - ┌─────────────────┐ - │ Git Repos │ - │ (code + beads) │ - └─────────────────┘ -``` - -### CLI Commands - -```bash -gt outpost list # List configured outposts -gt outpost status [name] # Detailed status -gt outpost add ... # Add new outpost -gt outpost ping # Test connectivity -``` - -### Implementation Status - -Federation is tracked in **gt-9a2** (P3 epic). Key tasks: -- `gt-9a2.1`: Outpost/Worker interfaces -- `gt-9a2.2`: LocalOutpost (refactor current spawning) -- `gt-9a2.5`: SSHOutpost -- `gt-9a2.8`: CloudRunOutpost - -## Implementation Status - -Gas Town is written in Go. Current development epics: - -- **Management**: gt-f9x (Town & Rig Management: install, doctor, federation) - -See beads issues with `bd list --status=open` for current work items. diff --git a/docs/beads-data-plane.md b/docs/beads-data-plane.md deleted file mode 100644 index aafc2a14..00000000 --- a/docs/beads-data-plane.md +++ /dev/null @@ -1,315 +0,0 @@ -# Beads as Data Plane (CLAW) - -> **Status**: Design documentation -> **See also**: [pinned-beads-design.md](pinned-beads-design.md), [propulsion-principle.md](propulsion-principle.md) - -## Overview - -Gas Town agents coordinate through **Beads** - a git-backed issue tracker. - -**CLAW**: **C**ommits as **L**edger of **A**ll **W**ork. Git commits are the persistence -mechanism. Every state change becomes a commit, giving us atomic updates, full history, -and distributed sync for free. - -We store agent state (work assignments, mail, molecules, hooks) as beads issues. - -## How We Use It - -We're treating beads as more than an issue tracker: - -- **Work molecules** are issues with steps as child issues -- **Mail messages** are issues with sender/recipient encoded in fields -- **Hooks** are queries over issues (`assignee = me AND pinned = true`) -- **Inboxes** are queries over issues (`assignee = me AND status = open AND has from: label`) - -Everything is an issue. The semantics come from how fields are used. - -## The Unified Data Model - -Every beads issue has these core fields: - -| Field | Type | Purpose | -|-------|------|---------| -| `id` | string | Unique identifier (e.g., `gt-xxxx`, `hq-yyyy`) | -| `title` | string | Brief summary | -| `description` | string | Full content | -| `status` | enum | `open`, `in_progress`, `closed`, `pinned` | -| `assignee` | string | Who this is assigned to | -| `priority` | int | 0=critical, 1=high, 2=normal, 3=low, 4=backlog | -| `type` | string | `task`, `bug`, `feature`, `epic` | -| `labels` | []string | Metadata tags | -| `pinned` | bool | Whether this is pinned to assignee's hook | -| `parent` | string | Parent issue ID (for hierarchies) | -| `created_at` | timestamp | Creation time | - -## Field Reuse Patterns - -### Work Molecules - -A **molecule** is the general abstraction for executable work in Gas Town. Molecules -can take various **shapes** - structural patterns that encode different workflow -semantics. The data plane stores all shapes the same way; the semantics come from -how the structure is interpreted. - -**Common molecular shapes:** - -| Shape | Structure | Use Case | -|-------|-----------|----------| -| **Epic** | Parent + flat children (TODO list) | Simple feature work, linear decomposition | -| **Christmas Ornament** | Trunk + dynamic arms + base | Patrol cycles with variable fanout | -| **Compound** | Multiple bonded molecules | Complex multi-phase workflows | -| **Polymer** | Chained compound molecules | Large-scale autonomous operation | - -> **All epics are molecules. Not all molecules are epics.** -> For the full taxonomy of shapes, see [molecular-chemistry.md](molecular-chemistry.md). - -Here's an **epic** (the simplest molecular shape) stored as beads data: - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Issue: gt-abc1 │ -│ ───────────────────────────────────────────────────────── │ -│ title: "Implement user authentication" │ -│ type: epic ← SHAPE: TODO LIST │ -│ assignee: "gastown/crew/max" │ -│ pinned: true ← ON MY HOOK │ -│ status: in_progress │ -│ description: "Full auth flow with OAuth..." │ -│ children: [gt-abc2, gt-abc3, gt-abc4] ← FLAT CHILDREN │ -└─────────────────────────────────────────────────────────────┘ -``` - -The hook query: `WHERE assignee = me AND pinned = true` - -**Why the data plane doesn't distinguish shapes:** The shape lives in the structure -(parent/child relationships, dependency edges), not in a separate field. An epic -is recognized by its flat parent-children structure. A Christmas Ornament is -recognized by dynamic arms bonded during execution. The beads database just stores -issues with relationships - the chemistry layer interprets the shape. - -### Mail Messages - -Mail reuses the same fields with different semantics: - -``` -MAIL FIELD → BEADS FIELD -═══════════════════════════════════════════════════════════ - -To (recipient) → assignee -Subject → title -Body → description -From (sender) → labels: ["from:mayor/"] -Thread ID → labels: ["thread:thread-xxx"] -Reply-To → labels: ["reply-to:hq-yyy"] -Message Type → labels: ["msg-type:task"] -Unread → status: open -Read → status: closed -``` - -Example mail message as a bead: - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Issue: hq-def2 │ -│ ───────────────────────────────────────────────────────── │ -│ title: "Fix the auth bug" ← SUBJECT │ -│ assignee: "gastown/crew/max" ← TO │ -│ status: open ← UNREAD │ -│ labels: ["from:mayor/", ← FROM │ -│ "thread:thread-abc", ← THREAD │ -│ "msg-type:task"] ← TYPE │ -│ description: "The OAuth flow is broken..." ← BODY │ -└─────────────────────────────────────────────────────────────┘ -``` - -The inbox query: `WHERE assignee = me AND status = open AND has_label("from:*")` - -### Distinguishing Mail from Work - -How does the system know if an issue is mail or work? - -| Indicator | Mail | Work | -|-----------|------|------| -| Has `from:` label | Yes | No | -| Has `pinned: true` | Rarely | Yes (when on hook) | -| Parent is molecule | No | Often | -| ID prefix | `hq-*` (town beads) | `gt-*` (rig beads) | - -The `from:` label is the canonical discriminator. Regular issues don't have senders. - -## Two-Tier Beads Architecture - -Gas Town uses beads at two levels, each with persistent and ephemeral components: - -``` -══════════════════════════════════════════════════════════════════════════ - TOWN LEVEL -══════════════════════════════════════════════════════════════════════════ - -┌─────────────────────────────────┐ ┌─────────────────────────────────┐ -│ PERSISTENT: ~/gt/.beads/ │ │ EPHEMERAL: ~/gt/.beads-wisp/ │ -│ ───────────────────────────── │ │ ───────────────────────────── │ -│ Prefix: hq-* │ │ Git tracked: NO │ -│ Git tracked: Yes │ │ Contains: │ -│ Contains: │ │ - Deacon patrol cycles │ -│ - All mail │ │ - Town-level ephemeral work │ -│ - Mayor coordination │ │ Lifecycle: │ -│ - Cross-rig work items │ │ Created → Executed → │ -│ Sync: Direct commit to main │ │ Squashed to digest → Deleted │ -└─────────────────────────────────┘ └─────────────────────────────────┘ - -══════════════════════════════════════════════════════════════════════════ - RIG LEVEL -══════════════════════════════════════════════════════════════════════════ - -┌─────────────────────────────────┐ ┌─────────────────────────────────┐ -│ PERSISTENT: /.beads/ │ │ EPHEMERAL: /.beads-wisp/ │ -│ ───────────────────────────── │ │ ───────────────────────────── │ -│ Prefix: gt-* (rig-specific) │ │ Git tracked: NO │ -│ Git tracked: Yes │ │ Contains: │ -│ Contains: │ │ - Witness patrol cycles │ -│ - Project issues │ │ - Refinery patrol cycles │ -│ - Molecules (work patterns) │ │ - Rig-level ephemeral work │ -│ - Agent hook states │ │ Lifecycle: │ -│ Sync: Via beads-sync branch │ │ Created → Executed → │ -│ │ │ Squashed to digest → Deleted │ -└─────────────────────────────────┘ └─────────────────────────────────┘ -``` - -Key points: -- **Each level has persistent + ephemeral storage** -- **Town persistent** (`~/gt/.beads/`) - mail, mayor work, cross-rig coordination -- **Town ephemeral** (`~/gt/.beads-wisp/`) - deacon patrols -- **Rig persistent** (`/.beads/`) - project issues, molecules, hooks -- **Rig ephemeral** (`/.beads-wisp/`) - witness/refinery patrols -- **Sync strategies**: Persistent beads sync via git; ephemeral wisps never sync - -## The Query Model - -Both hooks and inboxes are **views** (queries) over the flat beads collection: - -``` -BEADS DATABASE (flat collection) -══════════════════════════════════════════════════════════════ - -┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ -│ hq-aaa │ │ hq-bbb │ │ gt-xxx │ │ gt-yyy │ -│ mail │ │ mail │ │ task │ │ molecule │ -│ to: max │ │ to: joe │ │ assign: │ │ assign: │ -│ open │ │ closed │ │ max │ │ max │ -│ │ │ │ │ pinned: │ │ │ -│ ↓ INBOX │ │ │ │ true │ │ │ -│ │ │ │ │ ↓ HOOK │ │ │ -└──────────┘ └──────────┘ └──────────┘ └──────────┘ - - -INBOX QUERY (for max): - SELECT * FROM beads - WHERE assignee = 'gastown/crew/max' - AND status = 'open' - AND labels CONTAINS 'from:*' - → Returns: [hq-aaa] - - -HOOK QUERY (for max): - SELECT * FROM beads - WHERE assignee = 'gastown/crew/max' - AND pinned = true - → Returns: [gt-xxx] -``` - -There is no container. No inbox bead. No hook bead. Just queries over issues. - -## Session Cycling Through the Data Lens - -When an agent cycles (hands off to fresh session), the data model ensures continuity: - -### What Persists (in beads) - -| Data | Location | Survives Restart | -|------|----------|------------------| -| Pinned molecule | Rig beads | Yes | -| Handoff mail | Town beads | Yes | -| Issue state | Rig beads | Yes | -| Git commits | Git | Yes | - -### What Doesn't Persist - -| Data | Why Not | -|------|---------| -| Claude context | Cleared on restart | -| In-memory state | Process dies | -| Uncommitted changes | Not in git | -| Unflushed beads | Not synced | - -### The Cycle - -``` -END OF SESSION START OF SESSION -══════════════════ ══════════════════ - -1. Commit & push code 1. gt prime (loads context) - -2. bd sync (flush beads) 2. gt mol status - └─ Molecule state saved └─ Query: pinned = true - ↓ -3. gt handoff -s "..." -m "..." 3. Hook has molecule? - └─ Creates mail in town beads ├─ YES → Execute it - (assignee = self) └─ NO → Query inbox - ↓ -4. Session dies 4. Inbox has handoff mail? - ├─ YES → Read context - └─ NO → Wait for work -``` - -The **molecule is the source of truth** for what you're working on. -The **handoff mail is supplementary context** (optional but helpful). - -## Command to Data Mapping - -| Command | Data Operation | -|---------|----------------| -| `gt mol status` | Query: `assignee = me AND pinned = true` | -| `gt mail inbox` | Query: `assignee = me AND status = open AND from:*` | -| `gt mail read X` | Read issue X, no status change | -| `gt mail delete X` | Close issue X (status → closed) | -| `gt sling X [Y]` | Hook X to Y (or self), inject start prompt | -| `gt hook X` | Update X: `pinned = true` (assign without action) | -| `gt handoff X` | Hook X, restart session (GUPP kicks in) | -| `bd pin X` | Update X: `pinned = true` | -| `bd close X` | Update X: `status = closed` | - -## Why This Design? - -### 1. Single Source of Truth - -All agent state lives in beads. No separate databases, no config files, no -hidden state. If you can read beads, you can understand the entire system. - -### 2. Queryable Everything - -Hooks, inboxes, work queues - all are just queries. Want to find all blocked -work? Query. Want to see what's assigned to a role? Query. The data model -supports arbitrary views. - -### 3. Git-Native Persistence - -Beads syncs through git. This gives you: -- Version history -- Branch-based isolation -- Merge conflict resolution -- Distributed replication - -### 4. No Schema Lock-in - -New use cases emerge by convention, not schema changes. Mail was added by -reusing `assignee` for recipient and adding `from:` labels. No database -migration needed. - -## Related Documents - -- [pinned-beads-design.md](pinned-beads-design.md) - Hook semantics per role -- [propulsion-principle.md](propulsion-principle.md) - The "RUN IT" protocol -- [sling-design.md](sling-design.md) - Work assignment mechanics -- [molecular-chemistry.md](molecular-chemistry.md) - Full taxonomy of molecular shapes, phases, and operators -- [session-lifecycle.md](session-lifecycle.md) - The Single-Bond Principle and context cycling diff --git a/docs/bootstrap.md b/docs/bootstrap.md deleted file mode 100644 index 5543ca44..00000000 --- a/docs/bootstrap.md +++ /dev/null @@ -1,224 +0,0 @@ -# Bootstrapping Gas Town from an HQ - -This guide documents how to bootstrap a full Gas Town installation from an HQ repository (e.g., `steveyegge/stevey-gt`). - -## Prerequisites - -- macOS or Linux -- Git configured with SSH access to GitHub -- Homebrew (for macOS) - -## Overview - -A Gas Town HQ is a template repository containing: -- Town-level configuration (`mayor/`, `.beads/`) -- Rig configs (`gastown/config.json`, `beads/config.json`) -- CLAUDE.md for Mayor context - -The HQ does NOT contain: -- The actual gt binary (must be built) -- Full rig structures (must be populated) -- Agent state files (must be created) - -## Step 1: Clone the HQ - -```bash -git clone git@github.com:steveyegge/stevey-gt.git ~/gt -cd ~/gt -``` - -## Step 2: Install Go - -Gas Town is written in Go. Install it via Homebrew: - -```bash -brew install go -go version # Verify: should show go1.25+ -``` - -## Step 3: Clone Gastown into Mayor's Rig - -The gt binary lives in the gastown repository. Clone it into the mayor's rig directory: - -```bash -mkdir -p ~/gt/gastown/mayor -git clone git@github.com:steveyegge/gastown.git ~/gt/gastown/mayor/rig -``` - -## Step 4: Build the gt Binary - -```bash -cd ~/gt/gastown/mayor/rig -go build -o gt ./cmd/gt -``` - -Optionally install to PATH: - -```bash -mkdir -p ~/bin -cp gt ~/bin/gt -# Ensure ~/bin is in your PATH -``` - -## Step 5: Populate Rig Structures - -For each rig in your HQ (gastown, beads, etc.), create the full agent structure: - -### Gastown Rig - -```bash -cd ~/gt/gastown - -# Create directories -mkdir -p refinery witness polecats crew/main - -# Clone for refinery (canonical main) -git clone git@github.com:steveyegge/gastown.git refinery/rig - -# Clone for crew workspace -git clone git@github.com:steveyegge/gastown.git crew/main -``` - -### Beads Rig - -```bash -cd ~/gt/beads - -# Create directories -mkdir -p refinery mayor witness polecats crew/main - -# Clone for each agent -git clone git@github.com:steveyegge/beads.git refinery/rig -git clone git@github.com:steveyegge/beads.git mayor/rig -git clone git@github.com:steveyegge/beads.git crew/main -``` - -## Step 6: Create Agent State Files - -Each agent needs a state.json file: - -```bash -# Gastown agents -echo '{"role": "refinery", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/gastown/refinery/state.json -echo '{"role": "witness", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/gastown/witness/state.json -echo '{"role": "mayor", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/gastown/mayor/state.json - -# Beads agents -echo '{"role": "refinery", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/beads/refinery/state.json -echo '{"role": "witness", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/beads/witness/state.json -echo '{"role": "mayor", "last_active": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"}' > ~/gt/beads/mayor/state.json -``` - -## Step 7: Initialize Town-Level Beads - -```bash -cd ~/gt -bd init --prefix gm -``` - -If there are existing issues in JSONL that fail to import (e.g., due to invalid issue types), you can: -- Fix the JSONL manually and re-run `bd sync --import-only` -- Or start fresh with the empty database - -Run doctor to check for issues: - -```bash -bd doctor --fix -``` - -## Step 8: Verify Installation - -```bash -cd ~/gt - -# Check gt works -gt status - -# Check rig structure -gt rig list - -# Expected output: -# gastown - Polecats: 0, Crew: 1, Agents: [refinery mayor] -# beads - Polecats: 0, Crew: 1, Agents: [refinery mayor] -``` - -## Troubleshooting - -### Go not in PATH -If `go` command fails, ensure Homebrew's bin is in your PATH: -```bash -export PATH="/opt/homebrew/bin:$PATH" -``` - -### Witnesses Not Detected -Known issue (gm-2ej): The witness detection code checks for `witness/rig` but witnesses don't have a git clone. This is a bug in the detection logic - witnesses should still work. - -### Beads Import Fails -Known issue (gm-r6e): If your JSONL contains invalid issue types (e.g., "merge-request"), the import will fail. Either fix the JSONL or start with an empty database. - -## Full Bootstrap Script - -Here's a condensed script for bootstrapping: - -```bash -#!/bin/bash -set -e - -# Configuration -HQ_REPO="git@github.com:steveyegge/stevey-gt.git" -GASTOWN_REPO="git@github.com:steveyegge/gastown.git" -BEADS_REPO="git@github.com:steveyegge/beads.git" -TOWN_ROOT="$HOME/gt" - -# Clone HQ -git clone "$HQ_REPO" "$TOWN_ROOT" -cd "$TOWN_ROOT" - -# Install Go if needed -if ! command -v go &> /dev/null; then - brew install go -fi - -# Clone and build gastown -mkdir -p gastown/mayor -git clone "$GASTOWN_REPO" gastown/mayor/rig -cd gastown/mayor/rig -go build -o gt ./cmd/gt -cp gt ~/bin/gt -cd "$TOWN_ROOT" - -# Populate gastown rig -cd gastown -mkdir -p refinery witness polecats crew/main -git clone "$GASTOWN_REPO" refinery/rig -git clone "$GASTOWN_REPO" crew/main -echo '{"role": "refinery"}' > refinery/state.json -echo '{"role": "witness"}' > witness/state.json -echo '{"role": "mayor"}' > mayor/state.json -cd "$TOWN_ROOT" - -# Populate beads rig -cd beads -mkdir -p refinery mayor witness polecats crew/main -git clone "$BEADS_REPO" refinery/rig -git clone "$BEADS_REPO" mayor/rig -git clone "$BEADS_REPO" crew/main -echo '{"role": "refinery"}' > refinery/state.json -echo '{"role": "witness"}' > witness/state.json -echo '{"role": "mayor"}' > mayor/state.json -cd "$TOWN_ROOT" - -# Initialize beads -bd init --prefix gm - -# Verify -gt status -echo "Bootstrap complete!" -``` - -## Next Steps - -After bootstrapping: -1. Start a Mayor session: `gt mayor attach` -2. Check for work: `bd ready` -3. Spawn workers with molecules: `gt sling --molecule mol-shiny` diff --git a/docs/crew-tmux-config.md b/docs/crew-tmux-config.md deleted file mode 100644 index 5f50af92..00000000 --- a/docs/crew-tmux-config.md +++ /dev/null @@ -1,156 +0,0 @@ -# Crew tmux Configuration - -> **Status**: Personal Workflow -> **Location**: Your personal dotfiles (e.g., `~/.emacs.d/tmux.conf`) - -## Overview - -Crew workers are persistent identities. Unlike polecats (ephemeral, witness-managed), -crew members keep their names and workspaces across sessions. This makes them ideal -candidates for hardwired tmux keybindings. - -This document explains how to configure your personal tmux config to enable quick -cycling between crew sessions within a rig. - -## Why Personal Config? - -Crew session linking is **personal workflow**, not core Gas Town infrastructure: - -- Your crew members are stable identities you control -- The groupings reflect how *you* want to work -- Different users may have different crew setups -- Keeps Gas Town codebase focused on agent mechanics - -Store this in your personal dotfiles repo (e.g., `~/.emacs.d/`) for: -- Version control -- Sharing across machines -- Separation from project code - -## Setup - -### 1. Create tmux.conf in your dotfiles - -```bash -# Example: ~/.emacs.d/tmux.conf -``` - -### 2. Symlink from home - -```bash -ln -s ~/.emacs.d/tmux.conf ~/.tmux.conf -``` - -### 3. Configure crew cycling groups - -The key insight: use `run-shell` with a case statement to route `C-b n`/`C-b p` -based on the current session name. - -```tmux -# Crew session cycling - hardwired groups -# Group 1: gastown crew (max <-> joe) -# Group 2: beads crew (dave -> emma -> zoey -> dave) - -bind n run-shell ' \ - s="#{session_name}"; \ - case "$s" in \ - gt-gastown-crew-max) tmux switch-client -t gt-gastown-crew-joe ;; \ - gt-gastown-crew-joe) tmux switch-client -t gt-gastown-crew-max ;; \ - gt-beads-crew-dave) tmux switch-client -t gt-beads-crew-emma ;; \ - gt-beads-crew-emma) tmux switch-client -t gt-beads-crew-zoey ;; \ - gt-beads-crew-zoey) tmux switch-client -t gt-beads-crew-dave ;; \ - *) tmux switch-client -n ;; \ - esac' - -bind p run-shell ' \ - s="#{session_name}"; \ - case "$s" in \ - gt-gastown-crew-max) tmux switch-client -t gt-gastown-crew-joe ;; \ - gt-gastown-crew-joe) tmux switch-client -t gt-gastown-crew-max ;; \ - gt-beads-crew-dave) tmux switch-client -t gt-beads-crew-zoey ;; \ - gt-beads-crew-emma) tmux switch-client -t gt-beads-crew-dave ;; \ - gt-beads-crew-zoey) tmux switch-client -t gt-beads-crew-emma ;; \ - *) tmux switch-client -p ;; \ - esac' -``` - -### 4. Reload config - -```bash -tmux source-file ~/.tmux.conf -``` - -## Session Naming Convention - -Gas Town uses predictable session names: - -``` -gt--crew- -``` - -Examples: -- `gt-gastown-crew-max` -- `gt-gastown-crew-joe` -- `gt-beads-crew-dave` -- `gt-beads-crew-emma` -- `gt-beads-crew-zoey` - -This predictability enables hardwired keybindings. - -## Adding New Crew Members - -When you add a new crew member: - -1. Add entries to both `bind n` and `bind p` case statements -2. Maintain the cycle order (n goes forward, p goes backward) -3. Reload config: `tmux source-file ~/.tmux.conf` - -Example - adding `frank` to gastown crew: - -```tmux -# In bind n: -gt-gastown-crew-max) tmux switch-client -t gt-gastown-crew-joe ;; -gt-gastown-crew-joe) tmux switch-client -t gt-gastown-crew-frank ;; -gt-gastown-crew-frank) tmux switch-client -t gt-gastown-crew-max ;; - -# In bind p (reverse order): -gt-gastown-crew-max) tmux switch-client -t gt-gastown-crew-frank ;; -gt-gastown-crew-joe) tmux switch-client -t gt-gastown-crew-max ;; -gt-gastown-crew-frank) tmux switch-client -t gt-gastown-crew-joe ;; -``` - -## Fallback Behavior - -The `*) tmux switch-client -n ;;` fallback means: -- In a crew session → cycles within your group -- In any other session (mayor, witness, refinery) → standard all-session cycling - -This keeps the default behavior for non-crew contexts. - -## Starting Crew Sessions - -When starting crew sessions manually (not through Gas Town spawn), remember to -configure the status line: - -```bash -# Start session -tmux new-session -d -s gt--crew- -c /path/to/crew/ - -# Configure status (Gas Town normally does this automatically) -tmux set-option -t gt--crew- status-left-length 25 -tmux set-option -t gt--crew- status-left "👷 /crew/ " - -# Start Claude -tmux send-keys -t gt--crew- 'claude' Enter -``` - -## Tips - -- **Two-member groups**: For pairs like max/joe, n and p do the same thing (toggle) -- **Larger groups**: n cycles forward, p cycles backward -- **Mixed rigs**: Each rig's crew is a separate group - no cross-rig cycling -- **Testing**: Use `tmux display-message -p '#{session_name}'` to verify session names - -## Related - -- [session-lifecycle.md](session-lifecycle.md) - How sessions cycle -- [propulsion-principle.md](propulsion-principle.md) - The "RUN IT" protocol diff --git a/docs/cross-project-deps.md b/docs/cross-project-deps.md deleted file mode 100644 index 6806f300..00000000 --- a/docs/cross-project-deps.md +++ /dev/null @@ -1,248 +0,0 @@ -# Cross-Project Dependencies - -> Design for tracking dependencies across project boundaries without coupling to -> specific orchestrators. - -## Problem Statement - -When working on Gas Town, we frequently hit dependencies on Beads features (and -vice versa). Currently there's no formal mechanism to: - -1. Declare "I need capability X from project Y" -2. Signal "capability X is now available" -3. Park work waiting on external dependencies -4. Resume work when dependencies are satisfied - -## Design Principles - -1. **Beads-native**: The core mechanism lives in Beads, not Gas Town -2. **Orchestrator-agnostic**: Works with Gas Town, other orchestrators, or none -3. **Capability-based**: Reference published capabilities, not internal issue IDs -4. **Semaphore pattern**: Producer signals, consumers wait, no coordinator required - -## The Mechanism - -### Provider Side: Shipping Capabilities - -Projects declare and ship capabilities using labels: - -```bash -# Declare intent (optional, for visibility) -bd create --title="Add --assignee to mol run" \ - --add-label=export:mol-run-assignee - -# ... do work, close issue ... - -# Ship the capability (adds provides: label) -bd ship mol-run-assignee -``` - -The `bd ship` command: -- Finds issue with `export:mol-run-assignee` label -- Validates issue is closed (or `--force`) -- Adds `provides:mol-run-assignee` label -- Optionally notifies downstream (future) - -**Protected namespace**: `provides:*` labels can only be added via `bd ship`. - -### Consumer Side: Declaring Dependencies - -Projects declare external dependencies on capabilities: - -```bash -# At creation -bd create --title="Use --assignee in spawn.go" \ - --blocked-by="external:beads:mol-run-assignee" - -# Or later -bd update gt-xyz --blocked-by="external:beads:mol-run-assignee" -``` - -The `external:project:capability` syntax: -- `external:` - prefix indicating cross-project reference -- `project` - name from `external_projects` config -- `capability` - the capability name (matches `provides:X` label) - -### Configuration - -Projects configure paths to external projects: - -```yaml -# .beads/config.yaml -external_projects: - beads: ../beads - gastown: ../gastown - # Can also use absolute paths - other: /Users/steve/projects/other -``` - -### Resolution - -`bd ready` checks external dependencies: - -```bash -bd ready -# gt-xyz: blocked by external:beads:mol-run-assignee (not provided) -# gt-abc: ready - -# After beads ships the capability: -bd ready -# gt-xyz: ready -# gt-abc: ready -``` - -## Molecule Integration - -### Parking a Molecule - -When a polecat hits an external dependency mid-molecule: - -```bash -# Polecat discovers external dep not satisfied -gt park --step=gt-mol.3 --waiting="beads:mol-run-assignee" -``` - -This command: -1. Adds `blocked_by: external:beads:mol-run-assignee` to the step -2. Clears assignee on the step and molecule root -3. Sends handoff mail to self with context -4. Shuts down the polecat - -**"Parked" is a derived state**, not a new status: -- Molecule status: `in_progress` -- Molecule assignee: `null` -- Has step with unsatisfied external `blocked_by` - -### Querying Parked Work - -```bash -# Find parked molecules -bd list --status=in_progress --no-assignee --type=molecule - -# See what's blocked and on what -bd list --has-external-block -``` - -### Resuming Parked Work - -**Manual (launch):** -```bash -gt sling gt-mol-root -# Spawns polecat, which reads handoff mail and continues -``` - -**Automated (future):** -Deacon patrol checks parked molecules: -```yaml -- step: check-parked-molecules - action: | - For each molecule with status=in_progress, no assignee: - Check if external deps are satisfied - If yes: spawn polecat to resume -``` - -## Implementation Plan - -### Phase 1: Beads Core (bd-* issues) - -1. **bd ship command**: Add `provides:` label, protect namespace -2. **external: blocked_by**: Parse and store external references -3. **external_projects config**: Add to config schema -4. **bd ready resolution**: Check external deps via configured paths - -### Phase 2: Gas Town Integration (gt-* issues) - -1. **gt park command**: Set blocked_by, clear assignee, handoff, shutdown -2. **gt sling**: Resume parked molecule -3. **Patrol step**: Check parked molecules for unblocked - -### Phase 3: Automation (future) - -1. Push notifications on `bd ship` -2. Auto-resume via patrol -3. Cross-rig visibility in `gt status` - -## Examples - -### Full Flow: Adding --assignee to bd mol run - -**In beads repo:** -```bash -# Dave creates the issue -bd create --title="Add --assignee flag to bd mol run" \ - --type=feature \ - --add-label=export:mol-run-assignee - -# Dave implements, tests, closes -bd close bd-xyz - -# Dave ships the capability -bd ship mol-run-assignee -# Output: Shipped mol-run-assignee (bd-xyz) -``` - -**In gastown repo:** -```bash -# Earlier: Joe created dependent issue -bd create --title="Use --assignee in spawn.go" \ - --blocked-by="external:beads:mol-run-assignee" - -# bd ready showed it as blocked -bd ready -# gt-abc: blocked by external:beads:mol-run-assignee - -# After Dave ships: -bd ready -# gt-abc: ready - -# Joe picks it up -bd update gt-abc --status=in_progress --assignee=gastown/joe -``` - -### Parking Mid-Molecule - -```bash -# Polecat working on molecule, hits step 3 -# Step 3 needs beads:mol-run-assignee which isn't shipped - -gt park --step=gt-mol.3 --waiting="beads:mol-run-assignee" -# Setting blocked_by on gt-mol.3... -# Clearing assignee on gt-mol.3... -# Clearing assignee on gt-mol-root... -# Sending handoff mail... -# Polecat shutting down. - -# Later, after beads ships: -gt sling gt-mol-root -# Resuming molecule gt-mol-root... -# Reading handoff context... -# Continuing from step gt-mol.3 -``` - -## Design Decisions - -### Why capability labels, not issue references? - -Referencing `beads:bd-xyz` couples consumer to producer's internal tracking. -Referencing `beads:mol-run-assignee` couples to a published interface. - -The producer can refactor, split, or reimplement bd-xyz without breaking consumers. - -### Why "parked" as derived state? - -Adding a new status creates migration burden and complicates the state machine. -Deriving from existing fields (in_progress + no assignee + blocked) is simpler. - -### Why handoff via mail? - -Mail already handles context preservation. Parking is just a handoff to future-self -(or future-polecat). No new mechanism needed. - -### Why config-based project resolution? - -Alternatives considered: -- Git remote queries (complex, requires network) -- Hardcoded paths (inflexible) -- Central registry (single point of failure) - -Config is simple, explicit, and works offline. diff --git a/docs/deacon-plugins.md b/docs/deacon-plugins.md deleted file mode 100644 index f71e1ef2..00000000 --- a/docs/deacon-plugins.md +++ /dev/null @@ -1,289 +0,0 @@ -# Deacon Plugins - -Town-level plugins that run during the Deacon's patrol loop. - -## Overview - -The Deacon's patrol includes a `plugin-run` step. Rather than hardcoding what -runs, we use **directory-based discovery** at `~/gt/plugins/`. Each plugin -directory contains enough information for the Deacon to decide whether to run -it and what to do. - -Plugins are not pre-validated by tooling. The Deacon reads each plugin's -metadata and instructions, decides if the gate is open, and executes -accordingly. - -## Directory Structure - -``` -~/gt/plugins/ -├── beads-cleanup/ -│ ├── plugin.md # Gate info + instructions (combined) -│ └── state.json # Auto-managed runtime state -├── health-report/ -│ ├── plugin.md -│ └── state.json -└── wisp-pressure/ - ├── plugin.md - └── state.json -``` - -Each plugin is a directory. If the directory exists, the plugin exists. - -## Plugin Metadata (plugin.md) - -The `plugin.md` file contains YAML frontmatter for gate configuration, -followed by markdown instructions for the Deacon. - -```markdown ---- -gate: cooldown -interval: 24h -parallel: true ---- - -# Beads Cleanup - -Clean up old wisps daily. - -## Actions - -1. Run `bd cleanup --wisp --force` -2. Run `bd doctor` to verify health -3. Log summary of cleaned items -``` - -### Frontmatter Fields - -| Field | Required | Description | -|-------|----------|-------------| -| `gate` | Yes | Gate type: `cooldown`, `cron`, `condition`, `event` | -| `interval` | For cooldown | Duration: `1h`, `24h`, `5m`, etc. | -| `schedule` | For cron | Cron expression: `0 9 * * *` | -| `check` | For condition | Command that outputs a number | -| `operator` | For condition | Comparison: `gt`, `lt`, `eq`, `ge`, `le` | -| `threshold` | For condition | Numeric threshold | -| `trigger` | For event | Event name: `startup`, `heartbeat` | -| `cooldown` | Optional | Min time between runs (for condition gates) | -| `parallel` | Optional | If true, can run concurrently with other plugins | - -## Gate Types - -### Cooldown - -Run at most once per interval since last successful run. - -```yaml ---- -gate: cooldown -interval: 24h ---- -``` - -### Cron - -Run on a schedule (standard cron syntax). - -```yaml ---- -gate: cron -schedule: "0 9 * * *" # 9am daily ---- -``` - -### Condition - -Run when a command's output crosses a threshold. - -```yaml ---- -gate: condition -check: "bd count --type=wisp" -operator: gt -threshold: 50 -cooldown: 30m # Don't spam even if condition stays true ---- -``` - -The `check` command must output a single number. The Deacon compares it -against `threshold` using `operator`. - -### Event - -Run in response to specific events. - -```yaml ---- -gate: event -trigger: startup # Run once when daemon starts ---- -``` - -Triggers: `startup`, `heartbeat` (every patrol), `mail` (when inbox has items). - -## State Management - -The Deacon maintains `state.json` in each plugin directory: - -```json -{ - "lastRun": "2025-12-21T10:00:00Z", - "lastResult": "success", - "runCount": 42, - "nextEligible": "2025-12-22T10:00:00Z" -} -``` - -This is auto-managed. Plugins don't need to touch it. - -## Parallel Execution - -Plugins marked `parallel: true` can run concurrently. During `plugin-run`, -the Deacon: - -1. Scans `~/gt/plugins/` for plugins with open gates -2. Groups them: parallel vs sequential -3. Launches parallel plugins using Task tool subagents -4. Runs sequential plugins one at a time -5. Waits for all to complete before continuing patrol - -Sequential plugins (default) run in directory order. Use sequential for -plugins that modify shared state or have ordering dependencies. - -## Example Plugins - -### beads-cleanup (daily maintenance) - -```markdown ---- -gate: cooldown -interval: 24h -parallel: true ---- - -# Beads Cleanup - -Daily cleanup of wisp storage. - -## Actions - -1. Run `bd cleanup --wisp --force` to remove old wisps -2. Run `bd doctor` to check for issues -3. Report count of cleaned items -``` - -### wisp-pressure (condition-triggered) - -```markdown ---- -gate: condition -check: "bd count --type=wisp" -operator: gt -threshold: 100 -cooldown: 1h -parallel: true ---- - -# Wisp Pressure Relief - -Triggered when wisp count gets too high. - -## Actions - -1. Run `bd cleanup --wisp --age=4h` (remove wisps > 4h old) -2. Check `bd count --type=wisp` -3. If still > 50, run `bd cleanup --wisp --age=1h` -4. Report final count -``` - -### health-report (scheduled) - -```markdown ---- -gate: cron -schedule: "0 9 * * *" -parallel: false ---- - -# Morning Health Report - -Generate daily health summary for the overseer. - -## Actions - -1. Run `gt status` to get overall health -2. Run `bd stats` to get metrics -3. Check for stale agents (no heartbeat > 1h) -4. Send summary: `gt mail send --human -s "Morning Report" -m "..."` -``` - -### startup-check (event-triggered) - -```markdown ---- -gate: event -trigger: startup -parallel: false ---- - -# Startup Verification - -Run once when the daemon starts. - -## Actions - -1. Verify `gt doctor` passes -2. Check all rigs have healthy Witnesses -3. Report any issues to Mayor inbox -``` - -## Patrol Integration - -The `mol-deacon-patrol` step 3 (plugin-run) works as follows: - -```markdown -## Step: plugin-run - -Execute registered plugins whose gates are open. - -1. Scan `~/gt/plugins/` for plugin directories -2. For each plugin, read `plugin.md` frontmatter -3. Check if gate is open (based on type and state.json) -4. Collect eligible plugins into parallel and sequential groups - -5. For parallel plugins: - - Use Task tool to spawn subagents for each - - Each subagent reads the plugin's instructions and executes - - Wait for all to complete - -6. For sequential plugins: - - Execute each in order - - Read instructions, run actions, update state - -7. Update state.json for each completed plugin -8. Log summary: "Ran N plugins (M parallel, K sequential)" -``` - -## Limitations - -- **No polecat spawning**: Plugins cannot spawn polecats. If a plugin tries - to use `gt sling`, behavior is undefined. This may change in the future. - -- **No cross-plugin dependencies**: Plugins don't declare dependencies on - each other. If ordering matters, mark both as `parallel: false`. - -- **No plugin-specific state API**: Plugins can write to their own directory - if needed, but there's no structured API for plugin state beyond what the - Deacon auto-manages. - -## CLI Commands (Future) - -```bash -gt plugins list # Show all plugins, gates, status -gt plugins check # Show plugins with open gates -gt plugins show # Detail view of one plugin -gt plugins run # Force-run (ignore gate) -``` - -These commands are not yet implemented. The Deacon reads the directory -structure directly during patrol. diff --git a/docs/design/account-management.md b/docs/design/account-management.md deleted file mode 100644 index ce744f80..00000000 --- a/docs/design/account-management.md +++ /dev/null @@ -1,217 +0,0 @@ -# Gas Town Account Management Design - -## Problem Statement - -Claude Code users with multiple accounts (e.g., personal, work, team) need to: -1. Switch between accounts easily when quotas are exhausted -2. Set a global default account for all agents -3. Override per-spawn or per-role as needed - -## Current State - -- Claude Code stores config in `~/.claude/` and `~/.claude.json` -- Auth tokens stored in system keychain -- `CLAUDE_CONFIG_DIR` env var controls config location -- `/login` command switches accounts but overwrites in-place -- No built-in profile/multi-account support - -## Proposed Solution - -### Core Mechanism - -Each registered account gets its own config directory: - -``` -~/.claude-accounts/ -├── yegge/ # steve.yegge@gmail.com -│ ├── .claude/ # Full Claude Code config -│ └── .claude.json # Root config file -├── ghosttrack/ # steve@ghosttrack.com -│ ├── .claude/ -│ └── .claude.json -└── default -> ghosttrack/ # Symlink to current default -``` - -### Town Configuration - -File: `~/gt/mayor/accounts.json` - -This follows the existing pattern where town-level config lives in `mayor/`. - -```json -{ - "version": 1, - "accounts": { - "yegge": { - "email": "steve.yegge@gmail.com", - "description": "Personal/Gmail account", - "config_dir": "~/.claude-accounts/yegge" - }, - "ghosttrack": { - "email": "steve@ghosttrack.com", - "description": "Ghost Track business account", - "config_dir": "~/.claude-accounts/ghosttrack" - } - }, - "default": "ghosttrack" -} -``` - -### Environment Variable: GT_ACCOUNT - -Highest priority override. Set this to use a specific account: - -```bash -export GT_ACCOUNT=yegge -gt sling gastown # Uses yegge account -``` - -### Command Interface - -#### Account Management - -```bash -# List registered accounts -gt account list -# Output: -# yegge steve.yegge@gmail.com (personal) -# * ghosttrack steve@ghosttrack.com (default) - -# Add new account (creates config dir, prompts login) -gt account add [email] -gt account add work steve@company.com - -# Set default -gt account default -gt account default ghosttrack - -# Remove account (keeps config dir by default) -gt account remove -gt account remove yegge --delete-config - -# Check current/status -gt account status -# Output: -# Current: ghosttrack (steve@ghosttrack.com) -# GT_ACCOUNT env: not set -``` - -#### Spawn/Attach with Account Override - -```bash -# Override for a specific spawn -gt sling gastown --account=yegge - -# Override for crew attach -gt crew at --account=ghosttrack max - -# With env var (highest precedence) -GT_ACCOUNT=yegge gt sling gastown -``` - -### Implementation Details - -#### Account Resolution Order - -1. `GT_ACCOUNT` environment variable (highest) -2. `--account` flag on command -3. `default` in accounts.yaml (lowest) - -#### How Spawning Works - -When `gt sling` or `gt crew at` runs Claude Code: - -```go -func resolveAccountConfigDir() string { - // Check env var first - if handle := os.Getenv("GT_ACCOUNT"); handle != "" { - return getAccountConfigDir(handle) - } - - // Check flag - if handle := flags.Account; handle != "" { - return getAccountConfigDir(handle) - } - - // Use default from config - return getAccountConfigDir(config.Default) -} - -func spawnClaudeCode(workdir string, account string) { - configDir := resolveAccountConfigDir() - - cmd := exec.Command("claude", args...) - cmd.Env = append(os.Environ(), - fmt.Sprintf("CLAUDE_CONFIG_DIR=%s", configDir), - ) - // ... -} -``` - -#### Account Login Flow - -```bash -gt account add ghosttrack steve@ghosttrack.com -``` - -1. Creates `~/.claude-accounts/ghosttrack/` -2. Sets `CLAUDE_CONFIG_DIR` and runs `claude` -3. User completes `/login` with their account -4. Adds entry to `accounts.yaml` - -### Security Considerations - -- **No secrets in accounts.yaml** - Only handles and email addresses -- **Auth tokens in keychain** - Claude Code handles this per-config-dir -- **Config dir permissions** - Should be user-readable only - -### Future Extensions - -1. **Usage tracking** - `gt account status --usage` to show quota info -2. **Auto-switching** - When one account hits limits, prompt to switch -3. **Per-role defaults** - Different accounts for different roles: - ```yaml - role_defaults: - witness: yegge # Long-running patrol uses less quota - refinery: ghosttrack - ``` - -4. **API key accounts** - For when we support direct API access: - ```yaml - accounts: - api-team: - type: api_key - key_ref: GT_API_KEY # Env var containing key - ``` - -## Migration Path - -### Immediate (Manual) - -Users can start using separate config dirs today: - -```bash -# Set up account directories -mkdir -p ~/.claude-accounts/ghosttrack -export CLAUDE_CONFIG_DIR=~/.claude-accounts/ghosttrack -claude # Login as ghosttrack -``` - -### Phase 1: Basic Support - -- Add `accounts.json` parsing -- Add `gt account` subcommands -- Wire up `GT_ACCOUNT` env var in spawn - -### Phase 2: Full Integration - -- Add `--account` flags to all relevant commands -- Add status/usage tracking -- Add per-role defaults - -## Testing Plan - -1. Create test accounts config -2. Verify spawn uses correct config dir -3. Test override precedence -4. Test `gt account add` login flow diff --git a/docs/design/tmux-theming.md b/docs/design/tmux-theming.md deleted file mode 100644 index 99baa66f..00000000 --- a/docs/design/tmux-theming.md +++ /dev/null @@ -1,472 +0,0 @@ -# Design: Tmux Status Bar Theming (gt-vc1n) - -## Problem - -All Gas Town tmux sessions look identical: -- Same green/black status bars everywhere -- Hard to tell which rig you're in at a glance -- Session names get truncated (only 10 chars visible) -- No visual indication of worker role (polecat vs crew vs mayor) - -Current state: -``` -[gt-gastown] 0:zsh* "pane_title" 14:30 19-Dec -[gt-gastown] 0:zsh* "pane_title" 14:30 19-Dec <- which worker? -[gt-mayor] 0:zsh* "pane_title" 14:30 19-Dec -``` - -## Solution - -Per-rig color themes applied when tmux sessions are created, with optional user customization. - -### Goals -1. Each rig has a distinct color theme -2. Colors are automatically assigned from a predefined palette -3. Users can override colors per-rig -4. Status bar shows useful context (rig, worker, role) - -## Design - -### 1. Color Palette - -A curated palette of distinct, visually appealing color pairs (bg/fg): - -```go -// internal/tmux/theme.go -var DefaultPalette = []Theme{ - {Name: "ocean", BG: "#1e3a5f", FG: "#e0e0e0"}, // Deep blue - {Name: "forest", BG: "#2d5a3d", FG: "#e0e0e0"}, // Forest green - {Name: "rust", BG: "#8b4513", FG: "#f5f5dc"}, // Rust/brown - {Name: "plum", BG: "#4a3050", FG: "#e0e0e0"}, // Purple - {Name: "slate", BG: "#4a5568", FG: "#e0e0e0"}, // Slate gray - {Name: "ember", BG: "#b33a00", FG: "#f5f5dc"}, // Burnt orange - {Name: "midnight", BG: "#1a1a2e", FG: "#c0c0c0"}, // Dark blue-black - {Name: "wine", BG: "#722f37", FG: "#f5f5dc"}, // Burgundy - {Name: "teal", BG: "#0d5c63", FG: "#e0e0e0"}, // Teal - {Name: "copper", BG: "#6d4c41", FG: "#f5f5dc"}, // Warm brown -} -``` - -Palette criteria: -- Distinct from each other (no two look alike) -- Readable (sufficient contrast) -- Professional (no neon/garish colors) -- Dark backgrounds (easier on eyes in terminals) - -### 2. Configuration - -#### Per-Rig Config Extension - -Extend `RigConfig` in `internal/config/types.go`: - -```go -type RigConfig struct { - Type string `json:"type"` - Version int `json:"version"` - MergeQueue *MergeQueueConfig `json:"merge_queue,omitempty"` - Theme *ThemeConfig `json:"theme,omitempty"` // NEW -} - -type ThemeConfig struct { - // Name picks from palette (e.g., "ocean", "forest") - Name string `json:"name,omitempty"` - - // Custom overrides the palette with specific colors - Custom *CustomTheme `json:"custom,omitempty"` -} - -type CustomTheme struct { - BG string `json:"bg"` // hex color or tmux color name - FG string `json:"fg"` -} -``` - -#### Town-Level Config (optional) - -Allow global palette override in `mayor/town.json`: - -```json -{ - "theme": { - "palette": ["ocean", "forest", "rust", "plum"], - "mayor_theme": "midnight" - } -} -``` - -### 3. Theme Assignment - -When a rig is added (or first session created), auto-assign a theme: - -```go -// internal/tmux/theme.go - -// AssignTheme picks a theme for a rig based on its name. -// Uses consistent hashing so the same rig always gets the same color. -func AssignTheme(rigName string, palette []Theme) Theme { - h := fnv.New32a() - h.Write([]byte(rigName)) - idx := int(h.Sum32()) % len(palette) - return palette[idx] -} -``` - -This ensures: -- Same rig always gets same color (deterministic) -- Different rigs get different colors (distributed) -- No persistent state needed for assignment - -### 4. Session Creation Changes - -Modify `tmux.NewSession` to accept optional theming: - -```go -// SessionOptions configures session creation. -type SessionOptions struct { - WorkDir string - Theme *Theme // nil = use default -} - -// NewSessionWithOptions creates a session with theming. -func (t *Tmux) NewSessionWithOptions(name string, opts SessionOptions) error { - args := []string{"new-session", "-d", "-s", name} - if opts.WorkDir != "" { - args = append(args, "-c", opts.WorkDir) - } - - if _, err := t.run(args...); err != nil { - return err - } - - // Apply theme - if opts.Theme != nil { - t.ApplyTheme(name, *opts.Theme) - } - - return nil -} - -// ApplyTheme sets the status bar style for a session. -func (t *Tmux) ApplyTheme(session string, theme Theme) error { - style := fmt.Sprintf("bg=%s,fg=%s", theme.BG, theme.FG) - _, err := t.run("set-option", "-t", session, "status-style", style) - return err -} -``` - -### 5. Status Line Format - -#### Static Identity (Left) - -```go -// SetStatusFormat configures the status line for Gas Town sessions. -func (t *Tmux) SetStatusFormat(session, rig, worker, role string) error { - // Format: [gastown/Rictus] polecat - left := fmt.Sprintf("[%s/%s] %s ", rig, worker, role) - - if _, err := t.run("set-option", "-t", session, "status-left-length", "40"); err != nil { - return err - } - return t.run("set-option", "-t", session, "status-left", left) -} -``` - -#### Dynamic Context (Right) - -The right side shows dynamic info that agents can update: - -``` -gt-70b3 | 📬 2 | 14:30 -``` - -Components: -- **Current issue** - what the agent is working on -- **Mail indicator** - unread mail count (hidden if 0) -- **Time** - simple clock - -Implementation via tmux environment variables + shell expansion: - -```go -// SetDynamicStatus configures the right side with dynamic content. -func (t *Tmux) SetDynamicStatus(session string) error { - // Use a shell command that reads from env vars we set - // Agents update GT_ISSUE, we poll mail count - // - // Format: #{GT_ISSUE} | 📬 #{mail_count} | %H:%M - // - // tmux can run shell commands in status-right with #() - right := `#(gt status-line --session=` + session + `) %H:%M` - - if _, err := t.run("set-option", "-t", session, "status-right-length", "50"); err != nil { - return err - } - return t.run("set-option", "-t", session, "status-right", right) -} -``` - -#### `gt status-line` Command - -A fast command for tmux to call every few seconds: - -```go -// cmd/statusline.go -func runStatusLine(cmd *cobra.Command, args []string) error { - session := cmd.Flag("session").Value.String() - - // Get current issue from tmux env - issue, _ := tmux.GetEnvironment(session, "GT_ISSUE") - - // Get mail count (fast - just counts files or queries beads) - mailCount := mail.UnreadCount(identity) - - // Build output - var parts []string - if issue != "" { - parts = append(parts, issue) - } - if mailCount > 0 { - parts = append(parts, fmt.Sprintf("📬 %d", mailCount)) - } - - fmt.Print(strings.Join(parts, " | ")) - return nil -} -``` - -#### Agent Updates Issue - -Agents call this when starting/finishing work: - -```bash -# When starting work on an issue -gt issue set gt-70b3 - -# When done -gt issue clear -``` - -Implementation: - -```go -// cmd/issue.go -func runIssueSet(cmd *cobra.Command, args []string) error { - issueID := args[0] - session := os.Getenv("TMUX_PANE") // or detect from GT_* vars - - return tmux.SetEnvironment(session, "GT_ISSUE", issueID) -} -``` - -#### Mayor-Specific Status - -Mayor gets a different right-side format: - -``` -5 polecats | 2 rigs | 📬 1 | 14:30 -``` - -```go -func runMayorStatusLine() { - polecats := countActivePolecats() - rigs := countActiveRigs() - mail := mail.UnreadCount("mayor/") - - var parts []string - parts = append(parts, fmt.Sprintf("%d polecats", polecats)) - parts = append(parts, fmt.Sprintf("%d rigs", rigs)) - if mail > 0 { - parts = append(parts, fmt.Sprintf("📬 %d", mail)) - } - fmt.Print(strings.Join(parts, " | ")) -} -``` - -#### Example Status Bars - -**Polecat working on issue:** -``` -[gastown/Rictus] polecat gt-70b3 | 📬 1 | 14:30 -``` - -**Crew worker, no mail:** -``` -[gastown/max] crew gt-vc1n | 14:30 -``` - -**Mayor overview:** -``` -[Mayor] coordinator 5 polecats | 2 rigs | 📬 2 | 14:30 -``` - -**Idle polecat:** -``` -[gastown/Wez] polecat | 14:30 -``` - -### 6. Integration Points - -#### Session Manager (session/manager.go) - -```go -func (m *Manager) Start(polecat string, opts StartOptions) error { - // ... existing code ... - - // Get theme from rig config - theme := m.getTheme() - - // Create session with theme - if err := m.tmux.NewSessionWithOptions(sessionID, tmux.SessionOptions{ - WorkDir: workDir, - Theme: theme, - }); err != nil { - return fmt.Errorf("creating session: %w", err) - } - - // Set status format - m.tmux.SetStatusFormat(sessionID, m.rig.Name, polecat, "polecat") - - // ... rest of existing code ... -} -``` - -#### Mayor (cmd/mayor.go) - -```go -func runMayorStart(cmd *cobra.Command, args []string) error { - // ... existing code ... - - // Mayor uses a special theme - theme := tmux.MayorTheme() // Gold/dark - distinguished - - if err := t.NewSessionWithOptions(MayorSessionName, tmux.SessionOptions{ - WorkDir: townRoot, - Theme: &theme, - }); err != nil { - return fmt.Errorf("creating session: %w", err) - } - - t.SetStatusFormat(MayorSessionName, "town", "mayor", "coordinator") - - // ... rest ... -} -``` - -#### Crew (cmd/crew.go) - -Similar pattern - get rig theme and apply. - -### 7. Commands - -#### `gt theme` - View/Set Themes - -```bash -# View current rig theme -gt theme -# Theme: ocean (bg=#1e3a5f, fg=#e0e0e0) - -# View available themes -gt theme --list -# ocean, forest, rust, plum, slate, ember, midnight, wine, teal, copper - -# Set theme for current rig -gt theme set forest - -# Set custom colors -gt theme set --bg="#2d5a3d" --fg="#e0e0e0" -``` - -#### `gt theme apply` - Apply to Running Sessions - -```bash -# Re-apply theme to all running sessions in this rig -gt theme apply -``` - -### 8. Backward Compatibility - -- Existing sessions without themes continue to work (they'll just have default green) -- New sessions get themed automatically -- Users can run `gt theme apply` to update running sessions - -## Implementation Plan - -### Phase 1: Core Infrastructure -1. Add Theme types to `internal/tmux/theme.go` -2. Add ThemeConfig to `internal/config/types.go` -3. Implement `AssignTheme()` function -4. Add `ApplyTheme()` to Tmux wrapper - -### Phase 2: Session Integration -5. Modify `NewSession` to accept SessionOptions -6. Update session.Manager.Start() to apply themes -7. Update cmd/mayor.go to theme Mayor session -8. Update cmd/crew.go to theme crew sessions - -### Phase 3: Static Status Line -9. Implement SetStatusFormat() for left side -10. Apply to all session creation points -11. Update witness.go, spawn.go, refinery, daemon - -### Phase 4: Dynamic Status Line -12. Add `gt status-line` command (fast, tmux-callable) -13. Implement mail count lookup (fast path) -14. Implement `gt issue set/clear` for agents to update current issue -15. Configure status-right to call `gt status-line` -16. Add Mayor-specific status line variant - -### Phase 5: Commands & Polish -17. Add `gt theme` command (view/set/apply) -18. Add config file support for custom themes -19. Documentation -20. Update CLAUDE.md with `gt issue set` guidance for agents - -## File Changes - -| File | Changes | -|------|---------| -| `internal/tmux/theme.go` | NEW - Theme types, palette, assignment | -| `internal/tmux/tmux.go` | Add ApplyTheme, SetStatusFormat, SetDynamicStatus | -| `internal/config/types.go` | Add ThemeConfig | -| `internal/session/manager.go` | Use themed session creation | -| `internal/cmd/mayor.go` | Apply Mayor theme + Mayor status format | -| `internal/cmd/crew.go` | Apply rig theme to crew sessions | -| `internal/cmd/witness.go` | Apply rig theme | -| `internal/cmd/spawn.go` | Apply rig theme | -| `internal/cmd/theme.go` | NEW - gt theme command | -| `internal/cmd/statusline.go` | NEW - gt status-line (tmux-callable) | -| `internal/cmd/issue.go` | NEW - gt issue set/clear | -| `internal/daemon/lifecycle.go` | Apply rig theme | -| `internal/refinery/manager.go` | Apply rig theme | -| `CLAUDE.md` (various) | Document `gt issue set` for agents | - -## Open Questions - -1. ~~**Should refinery/witness have distinct colors?**~~ **RESOLVED** - - Answer: Same as rig polecats, role shown in status-left - -2. **Color storage location?** - - Option A: In rig config.json (requires file write) - - Option B: In beads (config-as-data approach from gt-vc1n) - - Recommendation: Start with config.json for simplicity - -3. **Hex colors vs tmux color names?** - - Hex: More precise, but some terminals don't support - - Names: Limited palette, but universal support - - Recommendation: Support both, default to hex with true-color fallback - -4. **Status-line refresh frequency?** - - tmux calls `#()` commands every `status-interval` seconds (default 15) - - Trade-off: Faster = more responsive, but more CPU - - Recommendation: 5 seconds (`set -g status-interval 5`) - -## Success Criteria - -- [ ] Each rig has distinct status bar color -- [ ] Users can identify rig at a glance -- [ ] Status bar shows rig/worker/role clearly (left side) -- [ ] Current issue displayed when agent sets it -- [ ] Mail indicator shows unread count -- [ ] Mayor shows aggregate stats (polecats, rigs) -- [ ] Custom colors configurable per-rig -- [ ] Works with existing sessions after `gt theme apply` -- [ ] Agents can update issue via `gt issue set` diff --git a/docs/federation-design.md b/docs/federation-design.md deleted file mode 100644 index 5fc6a492..00000000 --- a/docs/federation-design.md +++ /dev/null @@ -1,472 +0,0 @@ -# Federation Architecture: Ultrathink - -## The Problem - -Gas Town needs to scale beyond a single machine: -- More workers than one machine can handle (RAM, CPU, context windows) -- Geographic distribution (workers close to data/services) -- Cost efficiency (pay-per-use vs always-on VMs) -- Platform flexibility (support various deployment targets) - -## Two Deployment Models - -### Model A: "Town Clone" (VMs) - -Clone the entire `~/gt` workspace to a remote VM. It runs like a regular Gas Town: - -``` -┌─────────────────────────────────────────┐ -│ GCE VM (or any Linux box) │ -│ │ -│ ~/gt/ # Full town clone │ -│ ├── config/ # Town config │ -│ ├── mayor/ # Mayor (or none) │ -│ ├── gastown/ # Rig with agents │ -│ │ ├── polecats/ # Workers here │ -│ │ ├── refinery/ │ -│ │ └── witness/ │ -│ └── beads/ # Another rig │ -│ │ -│ Runs autonomously, syncs via git │ -└─────────────────────────────────────────┘ -``` - -**Characteristics:** -- Full autonomy if disconnected -- Familiar model - it's just another Gas Town -- VM overhead (cost, management, always-on) -- Coarse-grained scaling (spin up whole VMs) -- Good for: always-on capacity, long-running work, full independence - -**Federation via:** -- Git sync for beads (already works) -- Extended mail routing (`vm1:gastown/polecat`) -- SSH for remote commands - -### Model B: "Cloud Run Workers" (Containers) - -Workers are stateless containers that wake on demand: - -``` -┌─────────────────────────────────────────┐ -│ Cloud Run Service: gastown-worker │ -│ │ -│ ┌────────────────────────────────┐ │ -│ │ Container Instance │ │ -│ │ - Claude Code + git │ │ -│ │ - HTTP endpoint for work │ │ -│ │ - Persistent volume mount │ │ -│ │ - Scales 0→N automatically │ │ -│ └────────────────────────────────┘ │ -│ │ -│ Zero cost when idle │ -│ Persistent connections keep warm │ -└─────────────────────────────────────────┘ -``` - -**Characteristics:** -- Pay-per-use (nearly free when idle) -- Scales elastically (0 to many workers) -- No VM management -- Stateless(ish) - needs fast bootstrap or persistent storage -- Good for: burst capacity, background work, elastic scaling - -**Key insight from your friend:** -Persistent connections solve the "zero to one" problem. Keep the connection open, container stays warm, subsequent requests are fast. This transforms Cloud Run from "cold functions" to "elastic workers." - -## Unified Abstraction: Outposts - -To support "however people want to do it," we need an abstraction that covers both models (and future ones like K8s, bare metal, etc.). - -### The Outpost Concept - -An **Outpost** is a remote compute environment that can run workers. - -```go -type Outpost interface { - // Identity - Name() string - Type() OutpostType // local, ssh, cloudrun, k8s - - // Capacity - MaxWorkers() int - ActiveWorkers() int - - // Worker lifecycle - Spawn(issue string, config WorkerConfig) (Worker, error) - Workers() []Worker - - // Health - Ping() error - - // Optional: Direct communication (VM outposts) - SendMail(worker string, msg Message) error -} - -type OutpostType string -const ( - OutpostLocal OutpostType = "local" - OutpostSSH OutpostType = "ssh" // Full VM clone - OutpostCloudRun OutpostType = "cloudrun" // Container workers - OutpostK8s OutpostType = "k8s" // Future -) -``` - -### Worker Interface - -```go -type Worker interface { - ID() string - Outpost() string - Status() WorkerStatus // idle, working, done, failed - Issue() string // Current issue being worked - - // For interactive outposts (local, SSH) - Attach() error // Connect to worker session - - // For all outposts - Logs() (io.Reader, error) - Stop() error -} -``` - -### Outpost Implementations - -#### LocalOutpost -- Current model: tmux panes on localhost -- Uses existing Connection interface (LocalConnection) -- Workers are tmux sessions - -#### SSHOutpost -- Full Gas Town clone on remote VM -- Uses SSHConnection for remote ops -- Workers are remote tmux sessions -- Town config replicated to VM - -#### CloudRunOutpost -- Workers are container instances -- HTTP/gRPC for work dispatch -- No tmux (stateless containers) -- Persistent connections for warmth - -## Cloud Run Deep Dive - -### Container Design - -```dockerfile -FROM golang:1.21 AS builder -# Build gt binary... - -FROM ubuntu:22.04 -# Install Claude Code -RUN npm install -g @anthropic-ai/claude-code - -# Install git, common tools -RUN apt-get update && apt-get install -y git - -# Copy gt binary -COPY --from=builder /app/gt /usr/local/bin/gt - -# Entrypoint accepts work via HTTP -COPY worker-entrypoint.sh /entrypoint.sh -ENTRYPOINT ["/entrypoint.sh"] -``` - -### HTTP Work Protocol - -``` -POST /work -{ - "issue_id": "gt-abc123", - "rig_url": "https://github.com/steveyegge/gastown", - "beads_url": "https://github.com/steveyegge/gastown", - "context": { /* optional hints */ } -} - -Response (streaming): -{ - "status": "working|done|failed", - "branch": "polecat/gt-abc123", - "logs": "...", - "pr_url": "..." // if created -} -``` - -### Persistent Connections - -The "zero to one" solution: -1. Mayor opens HTTP/2 connection to Cloud Run -2. Connection stays open (Cloud Run keeps container warm) -3. Send work requests over same connection -4. Container processes work, streams results back -5. On idle timeout, connection closes, container scales down -6. Next request: small cold start, but acceptable - -``` -┌──────────┐ ┌────────────────┐ -│ Mayor │────HTTP/2 stream───▶│ Cloud Run │ -│ │◀───results stream───│ Container │ -└──────────┘ └────────────────┘ - │ │ - │ Connection persists │ Container stays - │ for hours if needed │ warm while - │ │ connection open - ▼ ▼ - [New work requests go over same connection] -``` - -### Git in Cloud Run - -Options for code access: - -1. **Clone on startup** (slow, ~30s+ for large repos) - - Simple but adds latency - - Acceptable if persistent connection keeps container warm - -2. **Cloud Storage FUSE mount** (read-only) - - Mount bucket with repo snapshot - - Fast startup - - Read-only limits usefulness - -3. **Persistent volume** (Cloud Run now supports!) - - Attach Cloud Storage or Filestore volume - - Git clone persists across container restarts - - Best of both worlds - -4. **Shallow clone with depth** - - `git clone --depth 1` for speed - - Sufficient for most worker tasks - - Can fetch more history if needed - -**Recommendation:** Persistent volume with shallow clone. Container starts, checks if clone exists, pulls if yes, shallow clones if no. - -### Beads Sync in Cloud Run - -Workers need beads access. Options: - -1. **Clone beads repo at startup** - - Same as code: persistent volume helps - - `bd sync` before and after work - -2. **Beads as API** (future) - - Central beads server - - Workers query/update via HTTP - - More complex but cleaner for distributed - -3. **Beads in git (current)** - - Works today - - Worker clones .beads, does work, pushes - - Git handles conflicts - -**Recommendation:** Start with git-based beads. It works today and Cloud Run workers can push to the beads repo just like local workers. - -### Mail in Cloud Run - -For VM outposts, mail is filesystem-based. For Cloud Run: - -**Option A: No mail needed** -- Cloud Run workers are "fire and forget" -- Mayor pushes work via HTTP, gets results via HTTP -- Simpler model for stateless workers - -**Option B: Mail via git** -- Worker checks `mail/inbox.jsonl` in repo -- Rare for workers to need incoming mail -- Mostly they just do work and report results - -**Recommendation:** Start with Option A. Cloud Run workers receive work via HTTP, report via HTTP. Mail is for long-running stateful agents (Witness, Refinery), not burst workers. - -### Cost Model - -Cloud Run pricing (as of late 2024): -- CPU: ~$0.00002400/vCPU-second -- Memory: ~$0.00000250/GiB-second -- Requests: ~$0.40/million - -For a worker running 5 minutes (300s) with 2 vCPU, 4GB RAM: -- CPU: 300 × 2 × $0.000024 = $0.0144 -- Memory: 300 × 4 × $0.0000025 = $0.003 -- **Total: ~$0.017 per worker session** - -50 workers × 5 minutes each = ~$0.85 - -**Key insight:** When idle (connection closed, scaled to zero): **$0** - -Compare to a VM running 24/7: ~$50-200/month - -Cloud Run makes burst capacity essentially free when not in use. - -### Claude API in Cloud Run - -Workers need Claude API access: - -1. **API key in Secret Manager** - - Cloud Run mounts secret as env var - - Standard pattern - -2. **Workload Identity** (if using Vertex AI) - - Service account with Claude access - - No keys to manage - -3. **Rate limiting concerns** - - Many concurrent workers = many API calls - - May need to coordinate or queue - - Could use Mayor as API proxy (future) - -## Architecture Overview - -``` -┌─────────────────────────────────────────────────────────────┐ -│ MAYOR │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ Outpost Manager │ │ -│ │ - Tracks all registered outposts │ │ -│ │ - Routes work to appropriate outpost │ │ -│ │ - Monitors worker status across outposts │ │ -│ └──────────────────────────────────────────────────────┘ │ -│ │ │ │ │ -│ ▼ ▼ ▼ │ -│ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │ -│ │ Local │ │ SSH │ │ CloudRun │ │ -│ │ Outpost │ │ Outpost │ │ Outpost │ │ -│ └────┬─────┘ └────┬─────┘ └──────┬───────┘ │ -└───────┼──────────────┼──────────────────┼───────────────────┘ - │ │ │ - ▼ ▼ ▼ - ┌─────────┐ ┌─────────┐ ┌─────────────┐ - │ tmux │ │ SSH │ │ HTTP/2 │ - │ panes │ │sessions │ │ connections │ - └─────────┘ └─────────┘ └─────────────┘ - │ │ │ - ▼ ▼ ▼ - ┌─────────┐ ┌─────────┐ ┌─────────────┐ - │ Workers │ │ Workers │ │ Workers │ - │ (local) │ │ (VM) │ │ (containers)│ - └─────────┘ └─────────┘ └─────────────┘ - │ │ │ - └──────────────┼──────────────────┘ - ▼ - ┌─────────────────┐ - │ Git Repos │ - │ (beads sync) │ - │ (code repos) │ - └─────────────────┘ -``` - -## Configuration - -```yaml -# ~/gt/config/outposts.yaml -outposts: - # Always present - the local machine - - name: local - type: local - max_workers: 4 - - # VM with full Gas Town clone - - name: gce-worker-1 - type: ssh - host: 10.0.0.5 - user: steve - ssh_key: ~/.ssh/gce_worker - town_path: /home/steve/ai - max_workers: 8 - - # Cloud Run for burst capacity - - name: cloudrun-burst - type: cloudrun - project: my-gcp-project - region: us-central1 - service: gastown-worker - max_workers: 20 # Or unlimited with cost cap - cost_cap_hourly: 5.00 # Optional spending limit - -# Work assignment policy -policy: - # Try local first, then VM, then Cloud Run - default_preference: [local, gce-worker-1, cloudrun-burst] - - # Override for specific scenarios - overrides: - - condition: "priority >= P3" # Background work - prefer: cloudrun-burst - - condition: "estimated_duration > 30m" # Long tasks - prefer: gce-worker-1 -``` - -**When policy becomes a molecule**: Static policy handles simple preference-based routing. But when assignment requires cognition - analyzing work characteristics, checking outpost health in real-time, making cost/speed tradeoffs, or handling degraded outposts - escalate to `mol-outpost-assign`. The policy file declares preferences; the molecule executes intelligent assignment. See [Config vs Molecule](architecture.md#config-vs-molecule-when-to-use-which) for the general principle. - -## Implementation Phases - -### Phase 1: Outpost Abstraction (Local) -- Define Outpost/Worker interfaces -- Implement LocalOutpost (refactor current polecat spawning) -- Configuration file for outposts -- `gt outpost list`, `gt outpost status` - -### Phase 2: SSH Outpost (VMs) -- Implement SSHConnection (extends existing Connection interface) -- Implement SSHOutpost -- VM provisioning docs (Terraform examples) -- `gt outpost add ssh ...` -- Test with actual GCE VM - -### Phase 3: Cloud Run Outpost -- Define worker container image -- Implement CloudRunOutpost -- HTTP/2 work dispatch protocol -- Persistent connection management -- Cost tracking/limits -- `gt outpost add cloudrun ...` - -### Phase 4: Policy & Intelligence -- Smart assignment based on workload characteristics -- Cost optimization (prefer free capacity) -- Auto-scaling policies -- Dashboard for cross-outpost visibility - -## Key Design Decisions - -### 1. Outpost as First-Class Concept -Rather than baking in specific platforms (SSH, Cloud Run), model the abstraction. This gives flexibility for future platforms (K8s, bare metal, other clouds). - -### 2. Workers Are Ephemeral -Whether local tmux, VM process, or Cloud Run container - workers are spawned for work and can be terminated. Don't assume persistence. - -### 3. Git as Source of Truth -Code and beads always sync via git. This works regardless of where workers run. Even Cloud Run workers clone/pull from git. - -### 4. HTTP for Cloud Run Control Plane -For Cloud Run specifically, use HTTP for work dispatch. Don't try to make filesystem mail work across containers. Keep it simple. - -### 5. Local-First Default -Always try local workers first. Remote outposts are for overflow/burst, not primary capacity. This keeps latency low and costs down. - -### 6. Graceful Degradation -If Cloud Run is unavailable, fall back to VM. If VM is down, use local only. System works with any subset of outposts. - -## Open Questions - -1. **Long-running sessions**: Cloud Run has request timeout limits (configurable up to 60 min, maybe longer now?). How does this interact with long Claude sessions? - -2. **Context handoff**: If a Cloud Run worker's container restarts mid-task, how do we resume? Mail-to-self? Checkpoint to storage? - -3. **Refinery in Cloud Run**: Could the Refinery itself run as a Cloud Run service? Long-running connection for merge queue processing? - -4. **Witness in Cloud Run**: Worker monitoring from Cloud Run? Or does Witness need to be local/VM? - -5. **Multi-region**: Cloud Run in multiple regions for geographic distribution? How to coordinate? - -## Summary - -The Outpost abstraction lets Gas Town scale flexibly: - -| Outpost Type | Best For | Cost Model | Scaling | -|--------------|----------|------------|---------| -| Local | Development, primary work | Free (your machine) | Fixed | -| SSH/VM | Long-running, full autonomy | Always-on VM cost | Manual | -| Cloud Run | Burst, background, elastic | Pay-per-use | Auto | - -Cloud Run's persistent connections solve the cold start problem, making it viable for interactive-ish work. Combined with VMs for heavier work and local for development, this gives a flexible spectrum of compute options. - -The key insight: **don't pick one model, support both.** Let users configure their outposts based on their needs, budget, and scale requirements. diff --git a/docs/formula_evolution.md b/docs/formula_evolution.md deleted file mode 100644 index c0cf8a21..00000000 --- a/docs/formula_evolution.md +++ /dev/null @@ -1,248 +0,0 @@ -# Formula Evolution: Predictions and Future Directions - -> Written: 2024-12-26 (Day 5 of formulas, Day 10 of molecules) -> Status: Speculative - institutional knowledge capture - -## Context - -Gas Town's workflow system has evolved rapidly: -- **Day 0**: Molecules conceived as workflow instances -- **Day 5**: Formulas introduced as compile-time templates -- **Day 10**: Patrols, session persistence, bond operators, protomolecules - -We've essentially discovered molecular chemistry for workflows. This document captures predictions about where this goes next. - -## Current Architecture - -``` -Formulas (TOML) → compile → Molecules (runtime) - ↓ ↓ -Templates with Executable workflows -{{variables}} with state persistence - ↓ ↓ -Stored in Walked by patrols, -.beads/formulas/ survive restarts -``` - -## The Package Ecosystem Parallel - -Every successful package ecosystem evolved through similar phases: - -| Phase | npm | Maven | Go | Formulas (predicted) | -|-------|-----|-------|-----|---------------------| -| 1. Local files | `./lib/` | `lib/*.jar` | `vendor/` | `.beads/formulas/` | -| 2. Central registry | npmjs.org | Maven Central | proxy.golang.org | Mol Mall | -| 3. Namespacing | `@org/pkg` | `groupId:artifactId` | `github.com/org/pkg` | `@org/formula` | -| 4. Version locking | `package-lock.json` | `` in pom | `go.sum` | `formulas.lock`? | -| 5. Composition | peer deps, plugins | BOM, parent POMs | embedding | nesting, AOP | - -## Formula Resolution Hierarchy - -Like `$PATH` or module resolution, formulas should resolve through layers: - -``` -1. Project: ./.beads/formulas/ # Project-specific workflows -2. Rig: ~/gt//.formulas/ # Rig-wide (shared across clones) -3. Town: ~/gt/.formulas/ # Town-wide (mayor's standard library) -4. User: ~/.gastown/formulas/ # Personal collection -5. Mall: mol-mall.io/ # Published packages -``` - -**Resolution order**: project → rig → town → user → mall (first match wins) - -**Explicit scoping**: `@gastown/shiny` always resolves to Mol Mall's gastown org - -## Formula Combinators - -Current composition is implicit (nesting, `needs` dependencies). We should formalize **formula combinators** - higher-order operations on workflows: - -### Sequential Composition -``` -A >> B # A then B -release >> announce # Release, then announce -``` - -### Parallel Composition -``` -A | B # A and B concurrent (join before next) -(build-linux | build-mac | build-windows) >> package -``` - -### Conditional Composition -``` -A ? B : C # if A succeeds then B else C -test ? deploy : rollback -``` - -### Wrapping (AOP) -``` -wrap(A, with=B) # B's before/after around A's steps -wrap(deploy, with=compliance-audit) -``` - -### Injection -``` -inject(A, at="step-id", B) # Insert B into A at specified step -inject(polecat-work, at="issue-work", shiny) -``` - -The Shiny-wrapping-polecat example: a polecat runs `polecat-work` formula, but the "issue-work" step (where actual coding happens) gets replaced/wrapped with `shiny`, adding design-review-test phases. - -### Extension/Override -``` -B extends A { - override step "deploy" { ... } - add step "notify" after "deploy" { ... } -} -``` - -### Algebra Properties - -These combinators should satisfy algebraic laws: -- **Associativity**: `(A >> B) >> C = A >> (B >> C)` -- **Identity**: `A >> noop = noop >> A = A` -- **Parallel commutativity**: `A | B = B | A` (order doesn't matter) - -## Higher Abstractions (Speculative) - -### 1. Formula Algebras -Formal composition with provable guarantees: -```toml -[guarantees] -compliance = "all MRs pass audit step" -coverage = "test step runs before any deploy" -``` - -The system could verify these properties statically. - -### 2. Constraint Workflows -Declare goals, let the system derive steps: -```toml -[goals] -reviewed = true -tested = true -compliant = true -deployed_to = "production" - -# System generates: review >> test >> compliance >> deploy -``` - -### 3. Adaptive Formulas -Learn from execution history: -```toml -[adapt] -on_failure_rate = { threshold = 0.3, action = "add-retry" } -on_slow_step = { threshold = "5m", action = "parallelize" } -``` - -### 4. Formula Schemas/Interfaces -Type system for workflows: -```toml -implements = ["Reviewable", "Deployable"] - -# Reviewable requires: has step with tag "review" -# Deployable requires: has step with tag "deploy", depends on "test" -``` - -Enables: "Give me all formulas that implement Deployable" - -### 5. Meta-Formulas -Formulas that generate formulas: -```toml -[meta] -for_each = ["service-a", "service-b", "service-c"] -generate = "deploy-{{item}}" -template = "microservice-deploy" -``` - -## Mol Mall Architecture (Predicted) - -### Registry Structure -``` -mol-mall.io/ -├── @gastown/ # Official Gas Town formulas -│ ├── shiny -│ ├── polecat-work -│ └── release -├── @acme-corp/ # Enterprise org (private) -│ └── compliance-review -├── @linus/ # Individual publisher -│ └── kernel-review -└── community/ # Unscoped public formulas - └── towers-of-hanoi -``` - -### Package Manifest -```toml -# formula.toml or mol.toml -[package] -name = "shiny" -version = "1.2.0" -description = "Engineer in a Box" -authors = ["Gas Town "] -license = "MIT" - -[dependencies] -"@gastown/review-patterns" = "^2.0" - -[peer-dependencies] -# Must be provided by the consuming workflow -"@gastown/test-runner" = "*" -``` - -### Lock File -```toml -# formulas.lock -[[package]] -name = "@gastown/shiny" -version = "1.2.0" -checksum = "sha256:abc123..." -source = "mol-mall.io" - -[[package]] -name = "@gastown/review-patterns" -version = "2.1.3" -checksum = "sha256:def456..." -source = "mol-mall.io" -``` - -## Distribution Scenarios - -### Scenario 1: Celebrity Formulas -"Linus Torvalds' kernel review formula" - expert-curated workflows that encode deep domain knowledge. High signal, community trust. - -### Scenario 2: Enterprise Compliance -Internal company formula for compliance reviews. Private registry, mandatory inclusion in all MR workflows via policy. - -### Scenario 3: Standard Library -Gas Town ships `@gastown/*` formulas as blessed defaults. Like Go's standard library - high quality, well-maintained, always available. - -### Scenario 4: Community Ecosystem -Thousands of community formulas of varying quality. Need: ratings, downloads, verified publishers, security scanning. - -## Open Questions - -1. **Versioning semantics**: SemVer? How do breaking changes work for workflows? - -2. **Security model**: Formulas execute code. Sandboxing? Permissions? Signing? - -3. **Testing formulas**: How do you unit test a workflow? Mock steps? - -4. **Formula debugging**: Step through execution? Breakpoints? - -5. **Rollback semantics**: If step 7 of 10 fails, what's the undo story? - -6. **Cross-rig formulas**: Can a gastown formula reference a beads formula? - -7. **Formula inheritance depth**: How deep can `extends` chains go? Diamond problem? - -## Next Steps (Suggested) - -1. **Formalize combinator semantics** - Write spec for `>>`, `|`, `wrap`, `inject` -2. **Prototype Mol Mall** - Even a simple file-based registry to prove the model -3. **Add formula schemas** - `implements` field with interface definitions -4. **Build formula test harness** - Run formulas in dry-run/mock mode - ---- - -*This document captures thinking as of late December 2024. Formulas are evolving rapidly - update as we learn more.* diff --git a/docs/hq.md b/docs/hq.md deleted file mode 100644 index 3dee7900..00000000 --- a/docs/hq.md +++ /dev/null @@ -1,287 +0,0 @@ -# Gas Town HQ Design - -The **HQ** (headquarters) is the top-level directory where Gas Town is installed - the workspace that contains all your rigs, agents, and coordination infrastructure. - -## What Is an HQ? - -Think of HQ as the "mount point" for Gas Town. It's the root directory where: -- The Mayor operates from -- Rigs are registered and managed -- Town-level beads coordinate mail and handoffs -- The entire workspace is versioned as a git repository - -An HQ is NOT: -- A git clone of any project (rigs contain the clones) -- A hidden directory (it's visible and user-controlled) -- Tied to any specific project (it can manage multiple rigs) - -## HQ Structure - -``` -~/gt/ # HQ ROOT -├── .git/ # HQ is a git repo -├── .gitignore # Generated by gt git-init -├── .beads/ # Town-level beads (hq-* prefix) -│ ├── beads.db # Mayor mail, coordination, handoffs -│ └── config.yaml # Beads config with prefix: hq -│ -├── CLAUDE.md # Mayor role context (runs from here) -│ -├── mayor/ # Mayor config and state -│ ├── town.json # {"type": "town", "name": "..."} -│ ├── rigs.json # Registry of managed rigs -│ └── state.json # Mayor state -│ -├── rigs/ # Managed rig containers -│ ├── gastown/ # A rig (project container) -│ └── wyvern/ # Another rig -│ -└── / # OR rigs at HQ root (legacy) -``` - -## Creating an HQ - -Use `gt install` to create a new HQ: - -```bash -# Create a new HQ -gt install ~/gt - -# Create with git initialization -gt install ~/gt --git - -# Create and push to GitHub -gt install ~/gt --github=username/my-gastown --private - -# Initialize current directory as HQ -gt install . --name my-workspace -``` - -The install command: -1. Creates the directory structure (`mayor/`, `rigs/`) -2. Writes configuration files (`town.json`, `rigs.json`, `state.json`) -3. Generates `CLAUDE.md` with Mayor role context -4. Initializes town-level beads with `hq-` prefix -5. Optionally initializes git with `.gitignore` - -## HQ vs Town vs Rig - -| Concept | Description | Example | -|---------|-------------|---------| -| **HQ** | Installation directory | `~/gt/` | -| **Town** | Logical workspace (same as HQ) | The Gas Town instance | -| **Rig** | Project container within HQ | `~/gt/gastown/` | - -The terms "HQ" and "town" are often used interchangeably. An HQ IS a town. The distinction is physical (HQ = directory) vs logical (town = workspace concept). - -## Beads in an HQ - -An HQ has **two levels** of beads: - -### Town-Level Beads - -Located at `/.beads/` with `hq-` prefix: -- Mayor mail and inbox -- Cross-rig coordination messages -- Session handoff notes - -### Rig-Level Beads - -Each rig has its own `.beads/` with a project-specific prefix: -- Work issues (bugs, features, tasks) -- Merge requests -- Agent-local mail within the rig - -The Mayor sees both: town beads for mail, rig beads for work coordination. - -## Beads Redirect Pattern - -In complex setups, you may want the HQ root's `.beads/` to redirect to a rig's beads. This is useful when: -- Multiple systems share an HQ -- You want a single source of truth for beads -- Migration scenarios - -Create a redirect file: - -```bash -# Instead of .beads/ directory, create .beads/redirect file -mkdir .beads -echo "path/to/actual/.beads" > .beads/redirect -``` - -Example from a real setup: -``` -# ~/ai/.beads/redirect -# Redirect to gastown beads (Mayor workspace) -# The Mayor runs in ~/ai but manages gastown issues in mayor/rigs/gastown -mayor/rigs/gastown/.beads -``` - -**When to use redirects:** -- When rig beads should be the canonical town beads -- Hybrid setups where agents work in different locations - -## HQ Configuration Files - -### mayor/town.json - -Identifies this as a Gas Town installation: - -```json -{ - "type": "town", - "version": 1, - "name": "stevey-gastown", - "created_at": "2024-01-15T10:30:00Z" -} -``` - -### mayor/rigs.json - -Registry of managed rigs: - -```json -{ - "version": 1, - "rigs": { - "gastown": { - "git_url": "https://github.com/steveyegge/gastown", - "added_at": "2024-01-15T10:30:00Z" - }, - "wyvern": { - "git_url": "https://github.com/steveyegge/wyvern", - "added_at": "2024-01-16T09:00:00Z" - } - } -} -``` - -### mayor/state.json - -Mayor agent state: - -```json -{ - "role": "mayor", - "last_active": "2024-01-17T14:30:00Z" -} -``` - -## Git for HQs - -An HQ should be a git repository. This enables: -- Versioning of configuration -- Beads sync across machines -- Session handoff via beads commits -- Recovery after failures - -### Initialize git - -```bash -gt git-init # Basic git setup -gt git-init --github=user/repo # Create GitHub repo -gt git-init --github=user/repo --private # Private repo -``` - -### Standard .gitignore - -The `gt git-init` command creates: - -```gitignore -# Gas Town HQ gitignore - -# Agent sessions and logs -*.log -*.pid -/sessions/ - -# Rig working directories (managed separately) -/rigs/*/polecats/*/ -/rigs/*/refinery/rig/ -/rigs/*/crew/*/ - -# Sensitive files -.env -*.key -*.pem -credentials.json - -# Editor and OS -.DS_Store -*.swp -*~ -.idea/ -.vscode/ - -# Beads daemon -.beads/beads.sock -.beads/*.pid -``` - -## HQ Health Checks - -Run `gt doctor` to check HQ health: - -```bash -gt doctor # Check all -gt doctor --fix # Auto-fix issues -``` - -Checks include: -- Configuration file validity -- Mayor state consistency -- Rig registry accuracy -- Beads database health -- Git state cleanliness - -## HQ Templates - -For organizations wanting consistent Gas Town setups, create a template repository: - -```bash -# Create template HQ -gt install ~/gt-template --git --no-beads -# Customize CLAUDE.md, add standard rigs -# Push to GitHub as template repo - -# Users clone template -gh repo create my-gastown --template org/gt-template -cd my-gastown -gt install . --force # Reinitialize with fresh beads -``` - -## Migration Between HQs - -To move Gas Town to a new location: - -1. **Export beads state:** - ```bash - bd export > beads-backup.jsonl - ``` - -2. **Create new HQ:** - ```bash - gt install ~/new-hq --git - ``` - -3. **Add rigs:** - ```bash - cd ~/new-hq - gt rig add gastown https://github.com/user/gastown - ``` - -4. **Import beads:** - ```bash - cd ~/new-hq - bd import < beads-backup.jsonl - ``` - -## Summary - -| Action | Command | -|--------|---------| -| Create HQ | `gt install ` | -| Initialize git | `gt git-init` | -| Add rig | `gt rig add ` | -| Check health | `gt doctor` | -| View status | `gt status` | diff --git a/docs/merge-queue-design.md b/docs/merge-queue-design.md deleted file mode 100644 index cd7ae8c6..00000000 --- a/docs/merge-queue-design.md +++ /dev/null @@ -1,543 +0,0 @@ -# Merge Queue Design - -The merge queue coordinates landing completed work. MRs are ephemeral wisps (not synced beads), and polecat branches stay local (never pushed to origin). - -**Key insight**: Git is already a ledger. Beads track durable state (issues). MRs are transient operational state - perfect for wisps. - -## Architecture (Current) - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ LOCAL MERGE QUEUE │ -│ │ -│ Polecat worktree ──commit──► local branch (polecat/nux) │ -│ │ │ │ -│ │ │ (same .git) │ -│ ▼ ▼ │ -│ .beads-wisp/mq/mr-xxx.json Refinery worktree │ -│ │ │ │ -│ └──────────────────────────┘ │ -│ │ │ -│ Refinery reads MR, merges branch │ -│ │ │ -│ ▼ │ -│ git push origin main │ -│ │ │ -│ Delete local branch │ -│ Delete MR wisp file │ -└─────────────────────────────────────────────────────────────────┘ -``` - -**Key points:** -- Refinery is a worktree of the same git repo as polecats (shared .git) -- Polecat branches are local only - never pushed to origin -- MRs stored in `.beads-wisp/mq/` (ephemeral, not synced) -- Only `main` branch gets pushed to origin after merge -- Source issues tracked in beads (durable), closed after merge - -## Overview - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ MERGE QUEUE │ -│ │ -│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ -│ │ MR #1 │→ │ MR #2 │→ │ MR #3 │→ │ MR #4 │ │ -│ │ (ready) │ │(blocked) │ │ (ready) │ │(blocked) │ │ -│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ -│ ↓ ↓ │ -│ ┌─────────────────────────────────────────────────────┐ │ -│ │ ENGINEER (processes queue) │ │ -│ │ 1. bd ready --type=merge-request │ │ -│ │ 2. Process in priority order │ │ -│ │ 3. Merge, test, close or reject │ │ -│ └─────────────────────────────────────────────────────┘ │ -│ ↓ │ -│ ┌──────────┐ │ -│ │ main │ │ -│ └──────────┘ │ -└─────────────────────────────────────────────────────────────────┘ -``` - -## Merge Request Schema - -A merge request is a JSON file in `.beads-wisp/mq/`: - -```json -// .beads-wisp/mq/mr-1703372400-a1b2c3d4.json -{ - "id": "mr-1703372400-a1b2c3d4", - "branch": "polecat/nux", - "target": "main", - "source_issue": "gt-xyz", - "worker": "nux", - "rig": "gastown", - "title": "Merge: gt-xyz", - "priority": 1, - "created_at": "2025-12-23T20:00:00Z" -} -``` - -### ID Convention - -MR IDs follow the pattern: `mr--` - -Example: `mr-1703372400-a1b2c3d4` - -These are ephemeral - deleted after merge. Source issues in beads provide the durable record. - -### Creating Merge Requests - -Workers submit to the queue via: - -```bash -# Worker signals work is ready (preferred) -gt done # Auto-detects branch, issue, creates MR - -# Or explicit submission -gt mq submit # Auto-detects branch, issue, worker -gt mq submit --issue gt-xyz # Explicit issue - -# Under the hood, this writes to .beads-wisp/mq/: -# - Creates mr--.json -# - No beads issue created (MRs are ephemeral wisps) -# - Branch stays local (never pushed to origin) -``` - -## Queue Ordering - -The queue is ordered by: - -1. **Dependencies first** - If MR-B depends on MR-A, A merges first -2. **Priority** - P0 before P1 before P2 -3. **Age** - Older requests before newer (FIFO within priority) - -### Dependency-Based Ordering - -When workers complete related work, dependencies ensure correct order: - -``` -gt-epic (epic) -├── gt-epic.1 (task) → gt-mr-001 (merge request) -├── gt-epic.2 (task) → gt-mr-002 (depends on gt-mr-001) -└── gt-epic.3 (task) → gt-mr-003 (depends on gt-mr-002) -``` - -The Engineer queries `bd ready --type=merge-request` to find MRs with no unmerged dependencies. - -### Integration Branches - -For batch work on an epic, use an integration branch: - -``` - main - │ - ├──────────────────────────┐ - │ │ - integration/gt-epic (other work) - │ - ┌──────────┼──────────┐ - │ │ │ - MR #1 MR #2 MR #3 -``` - -Integration branch workflow: -1. Create integration branch from main: `integration/gt-epic` -2. MRs target the integration branch, not main -3. After all epic work merges, integration branch merges to main -4. Single PR for epic = easier review, atomic landing - -```bash -# Create integration branch for epic -gt mq integration create gt-epic - -# MRs auto-target integration branch -gt mq submit --epic gt-epic # Targets integration/gt-epic - -# Land entire epic to main -gt mq integration land gt-epic -``` - -## Engineer Processing Loop - -The Engineer (formerly Refinery) processes the merge queue continuously: - -``` -┌─────────────────────────────────────────────────────────────┐ -│ ENGINEER LOOP │ -│ │ -│ while true: │ -│ 1. ready_mrs = bd ready --type=merge-request │ -│ │ -│ 2. if ready_mrs.empty(): │ -│ sleep(poll_interval) │ -│ continue │ -│ │ -│ 3. mr = ready_mrs.first() # Highest priority, oldest │ -│ │ -│ 4. bd update mr.id --status=in_progress │ -│ │ -│ 5. result = process_merge(mr) │ -│ │ -│ 6. if result.success: │ -│ bd close mr.id --reason="merged: {sha}" │ -│ else: │ -│ handle_failure(mr, result) │ -│ │ -│ 7. update_source_issue(mr) │ -└─────────────────────────────────────────────────────────────┘ -``` - -### Process Merge Steps - -```python -def process_merge(mr): - # 1. Branch is already local (shared .git with polecats) - # No fetch needed - refinery worktree sees polecat branches directly - - # 2. Check for conflicts with target - conflicts = git_check_conflicts(mr.branch, mr.target) - if conflicts: - return Failure(reason="conflict", files=conflicts) - - # 3. Merge to local target - git checkout {mr.target} - git merge {mr.branch} --no-ff -m "Merge {mr.branch}: {mr.title}" - - # 4. Run tests (configurable) - if config.run_tests: - result = run_tests() - if result.failed: - git reset --hard HEAD~1 # Undo merge - return Failure(reason="tests_failed", output=result.output) - - # 5. Push to origin (only main goes to origin) - git push origin {mr.target} - - # 6. Clean up source branch (local delete only) - if config.delete_merged_branches: - git branch -D {mr.branch} # Local delete, not remote - - # 7. Remove MR wisp file - os.remove(.beads-wisp/mq/{mr.id}.json) - - return Success(merge_commit=git_rev_parse("HEAD")) -``` - -### Handling Failures - -| Failure | Action | -|---------|--------| -| **Conflict** | Assign back to worker, add `needs-rebase` label | -| **Tests fail** | Assign back to worker, add `needs-fix` label | -| **Build fail** | Assign back to worker, add `needs-fix` label | -| **Flaky test** | Retry once, then assign back | -| **Infra issue** | Retry with backoff, escalate if persistent | - -```bash -# On conflict, Engineer does: -bd update gt-mr-xxx --assignee=Nux --labels=needs-rebase -gt send gastown/Nux -s "Rebase needed: gt-mr-xxx" \ - -m "Your branch conflicts with main. Please rebase and resubmit." -``` - -### Conflict Resolution Strategies - -1. **Assign back to worker** (default) - Worker rebases and resubmits -2. **Auto-rebase** (configurable) - Engineer attempts `git rebase` automatically -3. **Semantic merge** (future) - AI-assisted conflict resolution - -```yaml -# rig config.json -{ - "merge_queue": { - "on_conflict": "assign_back", # or "auto_rebase" or "semantic" - "run_tests": true, - "delete_merged_branches": true, - "retry_flaky_tests": 1 - } -} -``` - -## CLI Commands - -### gt mq (merge queue) - -```bash -# Submit work to queue -gt mq submit # Auto-detect from current branch -gt mq submit --issue gt-xyz # Explicit issue -gt mq submit --epic gt-epic # Target integration branch - -# View queue -gt mq list # Show all pending MRs -gt mq list --ready # Show only ready-to-merge -gt mq list --mine # Show MRs for my work -gt mq status gt-mr-xxx # Detailed MR status - -# Integration branches -gt mq integration create gt-epic # Create integration/gt-epic -gt mq integration land gt-epic # Merge integration to main -gt mq integration status gt-epic # Show integration branch status - -# Admin/debug -gt mq retry gt-mr-xxx # Retry a failed MR -gt mq reject gt-mr-xxx --reason "..." # Reject an MR -gt mq reorder gt-mr-xxx --after gt-mr-yyy # Manual reorder -``` - -### Command Details - -#### gt mq submit - -```bash -gt mq submit [--branch BRANCH] [--issue ISSUE] [--epic EPIC] - -# Auto-detection logic: -# 1. Branch: current git branch -# 2. Issue: parse from branch name (polecat/Nux/gt-xyz → gt-xyz) -# 3. Epic: if issue has parent epic, offer integration branch - -# Creates merge-request bead and prints MR ID -``` - -#### gt mq list - -```bash -gt mq list [--ready] [--status STATUS] [--worker WORKER] - -# Output: -# ID STATUS PRIORITY BRANCH WORKER AGE -# gt-mr-001 ready P0 polecat/Nux/gt-xyz Nux 5m -# gt-mr-002 in_progress P1 polecat/Toast/gt-abc Toast 12m -# gt-mr-003 blocked P1 polecat/Capable/gt-def Capable 8m -# (waiting on gt-mr-001) -``` - -## Beads Query Patterns - -The merge queue is just queries over beads: - -```bash -# Ready to merge (no blockers, not in progress) -bd ready --type=merge-request - -# All open MRs -bd list --type=merge-request --status=open - -# MRs for a specific epic (via labels or source_issue parent) -bd list --type=merge-request --label=epic:gt-xyz - -# Recently merged -bd list --type=merge-request --status=closed --since=1d - -# MRs by worker -bd list --type=merge-request --assignee=Nux -``` - -## State Machine - -``` - ┌─────────────┐ - │ CREATED │ - │ (open) │ - └──────┬──────┘ - │ - ┌──────▼──────┐ - ┌───────│ READY │───────┐ - │ │ (open) │ │ - │ └──────┬──────┘ │ - │ │ │ - blocked by ┌─────▼─────┐ rejected - dependency │ PROCESSING│ (manual) - │ │(in_progress) │ - │ └─────┬─────┘ │ - │ │ │ - │ ┌────────┴────────┐ │ - │ │ │ │ - │ success failure - │ │ │ │ - │ ▼ ▼ │ - │ ┌───────┐ ┌─────────┐│ - │ │MERGED │ │ FAILED ││ - │ │(closed) │ (open) ││ - │ └───────┘ └────┬────┘│ - │ │ │ - │ resubmit │ - │ │ │ - └─────────────────┴─────────┘ -``` - -## Audit and Observability - -The merge queue creates a complete audit trail for all integrated work: - -| MQ Event | Record Created | -|----------|----------------| -| Merge request submitted | Work completion claim with author | -| Tests pass | Quality verification record | -| Refinery approves | Validation with reviewer attribution | -| Merge commit | Immutable integration record | -| Rejection | Feedback record with reason | - -Every merge creates an immutable record: -- Who did the work (author attribution) -- Who validated it (Refinery attestation) -- When it landed (timestamp) -- What changed (commit diff) - -This enables full work attribution and quality tracking across the swarm. - -## Configuration - -### Rig-Level Config - -```json -// /config.json -{ - "merge_queue": { - "enabled": true, - "target_branch": "main", - "integration_branches": true, - "on_conflict": "assign_back", - "run_tests": true, - "test_command": "go test ./...", - "delete_merged_branches": true, - "retry_flaky_tests": 1, - "poll_interval": "30s", - "max_concurrent": 1 - } -} -``` - -### Per-Epic Overrides - -```bash -# Create epic with custom merge config -bd create --type=epic --title="Risky refactor" \ - --body="merge_config: - run_tests: true - test_command: 'go test -race ./...' - on_conflict: assign_back" -``` - -## Direct Landing (Bypass Queue) - -For single-polecat work or emergencies, Mayor can bypass the queue: - -```bash -gt land --direct / - -# This: -# 1. Verifies polecat session is terminated -# 2. Checks git state is clean -# 3. Merges directly to main (no MR created) -# 4. Closes the source issue -# 5. Cleans up the branch -``` - -Direct landing skips the queue but still records the work in beads. - -## Failure Recovery - -### Engineer Crash - -If Engineer crashes mid-merge: -1. MR stays `in_progress` -2. On restart, Engineer queries `bd list --type=merge-request --status=in_progress` -3. For each, check git state and either complete or reset - -### Partial Merge - -If merge succeeds but push fails: -1. Engineer retries push with exponential backoff -2. If persistent failure, roll back local merge -3. Mark MR as failed with reason - -### Conflicting Merges - -If two MRs conflict: -1. First one wins (lands first) -2. Second gets conflict status -3. Worker rebases and resubmits -4. Dependencies prevent this for related work - -## Observability - -### Metrics - -- `mq_pending_count` - MRs waiting -- `mq_processing_time` - Time from submit to merge -- `mq_success_rate` - Merges vs rejections -- `mq_conflict_rate` - How often conflicts occur -- `mq_test_failure_rate` - Test failures - -### Logs - -``` -[MQ] Processing gt-mr-abc123 (priority=P0, age=5m) -[MQ] Fetching branch polecat/Nux/gt-xyz -[MQ] No conflicts detected -[MQ] Merging to main -[MQ] Running tests: go test ./... -[MQ] Tests passed (32s) -[MQ] Pushing to origin -[MQ] Merged: abc123def -[MQ] Closed gt-mr-abc123 (reason=merged) -[MQ] Closed source issue gt-xyz -``` - -### Dashboard - -``` -┌─────────────────────────────────────────────────────────────┐ -│ MERGE QUEUE STATUS │ -├─────────────────────────────────────────────────────────────┤ -│ Pending: 3 In Progress: 1 Merged (24h): 12 Failed: 2│ -├─────────────────────────────────────────────────────────────┤ -│ QUEUE: │ -│ ► gt-mr-004 P0 polecat/Nux/gt-xyz Processing... │ -│ gt-mr-005 P1 polecat/Toast/gt-abc Ready │ -│ gt-mr-006 P1 polecat/Capable/gt-def Blocked (005) │ -│ gt-mr-007 P2 polecat/Nux/gt-ghi Ready │ -├─────────────────────────────────────────────────────────────┤ -│ RECENT: │ -│ ✓ gt-mr-003 Merged 5m ago (12s processing) │ -│ ✓ gt-mr-002 Merged 18m ago (45s processing) │ -│ ✗ gt-mr-001 Failed 22m ago (conflict with main) │ -└─────────────────────────────────────────────────────────────┘ -``` - -## Implementation Phases - -### Phase 1: Schema & CLI (gt-kp2, gt-svi) -- Define merge-request type in beads (or convention) -- Implement `gt mq submit`, `gt mq list`, `gt mq status` -- Manual Engineer processing (no automation yet) - -### Phase 2: Engineer Loop (gt-3x1) -- Implement processing loop -- Conflict detection and handling -- Test execution -- Success/failure handling - -### Phase 3: Integration Branches -- `gt mq integration create/land/status` -- Auto-targeting based on epic -- Batch landing - -### Phase 4: Advanced Features -- Auto-rebase on conflict -- Semantic merge (AI-assisted) -- Parallel test execution -- Cross-rig merge coordination - -## Open Questions - -1. **Beads schema extension**: Should merge-request be a first-class beads type, or just a convention (type field in description)? - -2. **High-traffic rigs**: For very active rigs, should MRs go to a separate beads repo to reduce sync contention? - -3. **Cross-rig merges**: If work spans multiple rigs, how do we coordinate? Federation design needed. - -4. **Rollback**: If a merge causes problems, how do we track and revert? Need `gt mq revert` command. diff --git a/docs/molecular-chemistry.md b/docs/molecular-chemistry.md deleted file mode 100644 index a07794dc..00000000 --- a/docs/molecular-chemistry.md +++ /dev/null @@ -1,987 +0,0 @@ -# Molecular Chemistry: Work Composition in Gas Town - -> *"Work is fractal. Money is crystallized labor. Blockchain was the mechanism -> searching for its purpose."* - -Gas Town is a **work composition and execution engine**. This document describes -**MEOW** - the **M**olecular **E**xpression **O**f **W**ork - a chemical algebra -for expressing, instantiating, and executing work at any scale, from single tasks -to massive polymers that can grind through weekends of autonomous operation. - -**Core insight**: Structure is computation. Content is cognition. They're separate. - -## The Work Lifecycle: Rig → Cook → Run - -Gas Town work flows through three phases: - -``` -RIG ────→ COOK ────→ RUN -(source) (artifact) (execution) -``` - -| Phase | What Happens | Operator | Output | -|-------|--------------|----------|--------| -| **Rig** | Compose formulas (source level) | `extends`, `compose` | Compound Formula | -| **Cook** | Instantiate artifacts | `cook`, `pour`, `wisp` | Proto, Mol, Wisp | -| **Run** | Execute steps | (agent execution) | Work done | - -**Rig** is authoring time - writing and composing formula files (TOML preferred, JSON supported). -**Cook** is compile time - expanding macros, applying aspects, flattening to pure graphs. -**Run** is execution time - agents provide cognition for each step. - -## The Complete Artifact Graph - -``` - SOURCE LEVEL (Rig) - ══════════════════ - -Formula ─────rig─────→ Compound Formula - │ (extends, │ - │ compose) │ - └──────────┬────────────┘ - │ - cook - │ - ▼ - ARTIFACT LEVEL (Bond) - ════════════════════ - -Proto ──────bond─────→ Compound Proto - │ \ │ \ - │ \ │ \ -pour wisp pour wisp - │ \ │ \ - ▼ ▼ ▼ ▼ - Mol Wisp ────bond────→ Linked Work - │ │ - └───┬───┘ - │ - run - │ - ▼ - EXECUTION - ═════════ - - Steps complete - Work gets done - Digests created -``` - -## Two Composition Operators - -Gas Town has **two** composition operators at different abstraction levels: - -| Operator | Level | Inputs | When to Use | -|----------|-------|--------|-------------| -| **Rig** | Source | Formula + Formula | Authoring time, in TOML/JSON | -| **Bond** | Artifact | Proto/Mol/Wisp + any | Runtime, on cooked artifacts | - -**Rig** composes formulas (TOML/JSON with `extends`, `compose`). -**Bond** composes artifacts (cooked protos, running mols/wisps). - -This separation is key: rig for design-time composition, bond for runtime composition. - -## The Steam Engine Metaphor - -Gas Town is an engine. Engines do work and generate steam. - -``` -Claude = Fire (the energy source) -Claude Code = Steam Engine (harnesses the fire) -Gas Town = Steam Train (coordinates engines on tracks) -Beads = Railroad Tracks (the persistent ledger of work) -``` - -In our chemistry: -- **Formulas** are the secret recipe (source code for workflows) -- **Proto molecules** are the fuel (cooked templates, ready to instantiate) -- **Mols** are liquid work (flowing, dynamic, adapting as steps complete) -- **Wisps** are the steam (transient execution traces that rise and dissipate) -- **Digests** are the distillate (condensed permanent records of completed work) - -## Formulas: The Source Layer - -**Formulas** sit above protos in the artifact hierarchy. They're the source code - -TOML or JSON files that define workflows with composition operators. - -TOML is preferred for human-edited formulas (multi-line strings, comments): - -```toml -# shiny.formula.toml - a basic workflow -formula = "shiny" -description = "The canonical right way" -version = 1 - -[[steps]] -id = "design" -description = "Think carefully about architecture" - -[[steps]] -id = "implement" -needs = ["design"] - -[[steps]] -id = "review" -needs = ["implement"] - -[[steps]] -id = "test" -needs = ["review"] - -[[steps]] -id = "submit" -needs = ["test"] -``` - -Convert existing JSON formulas with `bd formula convert --all`. - -### Formula Composition (Rigging) - -Formulas compose at the source level using `extends` and `compose`: - -```toml -# shiny-enterprise.formula.toml -formula = "shiny-enterprise" -extends = ["shiny"] # Inherit from base formula - -[compose] -aspects = ["security-audit"] # Weave in cross-cutting concern - -[[compose.expand]] -target = "implement" -with = "rule-of-five" # Apply macro expansion -``` - -### Cooking: Formula → Proto - -The `cook` command flattens a formula into a pure proto: - -```bash -bd cook shiny-enterprise -# Cooking shiny-enterprise... -# ✓ Cooked proto: shiny-enterprise (30 steps) -``` - -Cooking pre-expands all composition - macros, aspects, branches, gates. -The result is a flat step graph with no interpretation needed at runtime. - -### Formula Types - -| Type | Purpose | Example | -|------|---------|---------| -| **workflow** | Standard work definition | shiny, patrol | -| **expansion** | Macro template | rule-of-five | -| **aspect** | Cross-cutting concern | security-audit | - -## The Three Phases of Matter - -Work in Gas Town exists in three phases, following the states of matter: - -| Phase | Name | State | Storage | Behavior | -|-------|------|-------|---------|----------| -| **Solid** | Proto | Frozen template | `.beads/` (template label) | Crystallized, immutable, reusable | -| **Liquid** | Mol | Flowing instance | `.beads/` | Dynamic, adapting, persistent | -| **Vapor** | Wisp | Ephemeral trace | `.beads-wisp/` | Transient, dissipates, operational | - -### Proto (Solid Phase) - -Protos or protomolecules are **frozen workflow patterns** - crystallized templates that -encode reusable work structures. They're the "molds" from which instances are cast. - -Protos are stored as beads issues with `labels: ["template"]` and structured step data: - -```yaml -# Example: shiny.formula.toml (source) → cooked proto (beads) -formula: shiny -description: The canonical right way -version: 1 -steps: - - id: design - description: Think carefully about architecture - - id: implement - needs: [design] - - id: review - needs: [implement] - - id: test - needs: [review] - - id: submit - needs: [test] -``` - -**Properties:** -- Considered immutable once cooked (frozen), though source formulas are editable -- Named (e.g., `shiny`, `rule-of-five`) -- Stored in permanent beads with `template` label -- Can be composed into larger protos via formula algebra (extends, compose) - -### Mol (Liquid Phase) - -Mols are **flowing work instances** - live executions that adapt as steps -complete, status changes, and work evolves. - -**Properties:** -- Identified by head bead ID (e.g., `bd-abc123`) -- Dynamic - steps transition through states -- Persistent - survives sessions, crashes, context compaction -- Auditable - full history in beads - -### Wisp (Vapor Phase) - -Wisps are **ephemeral execution traces** - the steam that rises during work -and dissipates when done. - -**Properties:** -- Stored in `.beads-wisp/` (gitignored, never synced) -- Single-cycle lifetime -- Either evaporates (burn) or condenses to digest (squash) -- Used for patrol cycles, operational loops, routine work - -## Phase Transition Operators - -Work transitions between phases through specific operators: - -``` - ┌─────────────────┐ - │ PROTO │ - │ (solid) │ - └────────┬────────┘ - │ - ┌──────────────┼──────────────┐ - │ │ │ - pour wisp distill - │ │ ↑ - ▼ ▼ │ - ┌───────────────┐ ┌───────────────┐ │ - │ MOL │ │ WISP │ │ - │ (liquid) │ │ (vapor) │ │ - └───────┬───────┘ └───────┬───────┘ │ - │ │ │ - squash squash │ - │ │ │ - ▼ ▼ │ - ┌───────────────┐ ┌───────────────┐ │ - │ DIGEST │ │ (evaporates) │ │ - │ (condensed) │ │ or burn │ │ - └───────────────┘ └───────────────┘ │ - │ │ - └────────────────────────────┘ - (experience crystallizes) -``` - -### Pour: Solid → Liquid - -**Pour** instantiates a proto into a persistent mol - like pouring molten metal -into a mold to create a solid casting that will flow through the workflow. - -```bash -bd pour mol-feature # Create mol from proto -bd pour mol-feature --var version=1.0 # With variable substitution -``` - -**Use cases:** -- Feature work that spans sessions -- Important work needing audit trail -- Anything you might need to reference later - -### Wisp: Solid → Vapor (Sublimation) - -**Wisp** instantiates a proto directly into vapor - sublimation that skips -the liquid phase for ephemeral, operational work. - -```bash -bd wisp mol-patrol # Create wisp from proto -bd wisp mol-health-check # Ephemeral operational task -``` - -**Use cases:** -- Patrol cycles (deacon, witness) -- Health checks and monitoring -- One-shot orchestration runs -- Routine operations with no audit value - -### Squash: Liquid/Vapor → Condensed - -**Squash** condenses work into a permanent digest - the outcome crystallizes -while the execution trace compresses or evaporates. - -```bash -bd mol squash bd-abc123 # Squash mol to digest -bd mol squash bd-abc123 --summary="Completed auth" # With summary -``` - -**For mols:** Creates digest in permanent beads, preserves full outcome -**For wisps:** Creates digest, deletes wisp (vapor condenses to residue) - -### Burn: Vapor → Nothing - -**Burn** discards a wisp without creating a digest - the steam simply -evaporates with no residue. - -```bash -bd mol burn wisp-123 # Discard without digest -``` - -**Use cases:** -- Routine patrol cycles with nothing notable -- Failed attempts that don't need recording -- Test runs - -### Distill: Liquid → Solid (Crystallization) - -**Distill** extracts a reusable proto from an existing mol or epic - the -reverse of pour. Experience crystallizes into a template. - -```bash -bd mol distill bd-abc123 --as "Release Workflow" -bd mol distill bd-abc123 --var feature=auth --var version=1.0 -``` - -**Process:** -1. Analyze the existing work structure -2. Extract the pattern (steps, dependencies) -3. Replace concrete values with `{{variable}}` placeholders -4. Crystallize as a new proto - -**Use cases:** -- Team develops a good workflow organically, wants to reuse it -- Capture tribal knowledge as executable templates -- Create starting points for similar future work - -## The Polymorphic Bond Operator - -**Bond** is Gas Town's polymorphic combiner for artifacts. It operates at the -artifact level (post-cooking), handling different operand types with phase-aware -behavior. - -### The Bond Table (Symmetric) - -| bond | Proto | Mol | Wisp | -|------|-------|-----|------| -| **Proto** | Compound Proto | Pour, attach | Wisp, attach | -| **Mol** | Pour, attach | Link via edges | Link via edges | -| **Wisp** | Wisp, attach | Link via edges | Link via edges | - -The table is symmetric: bonding A+B produces the same structure as B+A. - -**Bond** handles different operand types with different phase behaviors: - -### Bond: Proto + Proto → Compound Proto - -Two solid templates fuse into a larger solid template. - -```bash -bd mol bond mol-review mol-deploy --as "Review and Deploy" -``` - -Creates a compound proto that includes both workflows. The result is a -reusable template (solid phase). - -### Bond: Proto + Mol → Pour + Attach - -A solid template melts into an existing liquid workflow. - -```bash -bd mol bond mol-hotfix bd-feature-123 -``` - -The proto is instantiated (as liquid by default) and attached to the -existing mol. The new issues become part of the flowing work. - -### Bond: Proto + Wisp → Wisp + Attach - -A solid template sublimates into an existing vapor workflow. - -```bash -bd mol bond mol-extra-check wisp-patrol-456 -``` - -The proto is created as a wisp (following the target's phase) and attaches. - -### Bond: Mol + Mol → Compound Mol - -Two liquid workflows merge into a larger flowing structure. - -```bash -bd mol bond bd-feature-123 bd-related-456 -``` - -Links them via dependency edges. Both continue flowing. - -### Bond: Wisp + Wisp → Compound Wisp - -Two vapor traces merge into a larger ephemeral cloud. - -```bash -bd mol bond wisp-123 wisp-456 -``` - -### Phase Override Flags - -Bond's creation behavior can be overridden: - -```bash -# Force liquid when attaching to wisp (found something important!) -bd mol bond mol-critical-bug wisp-patrol --pour - -# Force vapor when attaching to mol (ephemeral diagnostic) -bd mol bond mol-temp-check bd-feature --wisp -``` - -| Flag | Effect | Use Case | -|------|--------|----------| -| `--pour` | Force creation as liquid | "This matters, persist it" | -| `--wisp` | Force creation as vapor | "This is ephemeral, let it evaporate" | - -### Cross-Phase Bonding - -What happens when you bond liquid and vapor directly? - -```bash -bd mol bond bd-feature-123 wisp-456 # Mol + Wisp -``` - -**Answer: Reference-only linking.** They connect via dependency edges but -stay in their respective stores. No phase change occurs - you're linking -across the phase boundary without forcing conversion. - -This enables patterns like: -- Patrol wisp discovers issue → creates liquid mol for the fix -- Feature mol needs diagnostic → creates vapor wisp for the check -- The reference survives even when the wisp evaporates (ID stable) - -## Agent Attachment: Hooks and Pins - -Agents need work attached to them. In Gas Town, this uses **hooks** and **pins**. - -### The Hook - -Each agent has a **hook** - an anchor point where work hangs. It's the -agent's "pinned bead" - the top of their inbox, the work they're focused on. - -```bash -bd hook # Show what's on my hook -bd hook --agent deacon # Show deacon's hook -``` - -**Hook states:** -- **Empty (naked)**: Agent awaiting work assignment -- **Occupied**: Agent has work to execute -- **Multiple**: Agent managing several concurrent mols (rare) - -### Pin: Attaching Work to Agents - -**Pin** attaches a mol to an agent's hook - the action of assigning work. - -```bash -bd pin bd-feature-123 # Pin to my hook -bd pin bd-feature-123 --for witness # Pin to specific agent's hook -bd unpin # Detach current work -``` - -**The Witness → Polecat flow:** - -```bash -# Witness assigns work to polecat -bd pour mol-feature # Create liquid mol -bd pin bd-abc123 --for polecat-ace # Hang on polecat's hook -gt nudge polecat-ace # Wake the polecat -``` - -### Wisps Don't Need Pinning - -Wisps are single-cycle and don't survive session boundaries in the -traditional sense. Agents hold them in working memory for one cycle: - -```bash -# Deacon creates patrol wisp (no pin needed) -bd wisp mol-deacon-patrol # Create vapor -# ... execute steps ... -bd mol squash --summary="..." # Condense and dissipate -# Loop -``` - -## The Epic-Mol Relationship - -Epics and mols are **isomorphic** but represent different mental models. - -### Epic: The Business View - -An epic is a **simple mol shape** - essentially a TODO list: - -``` -epic-root -├── child.1 -├── child.2 -├── child.3 -└── child.4 -(flat list, no sibling dependencies, execution order implicit) -``` - -**Properties:** -- One level of children via `.N` numbering -- No explicit serial/parallel encoding -- Human-readable, business-oriented -- The natural shape most humans create - -### Mol: The Chemistry View - -A mol is the **general case** - arbitrary graphs with explicit workflow -semantics: - -``` -mol-root -├── phase-A (epic) -│ ├── task.1 ───blocks──→ task.2 (serial) -│ └── task.3 (parallel with task.1) -├── phase-B (epic) ←───blocked-by─── phase-A -│ └── ... -└── standalone (fanout) -(arbitrary DAG, explicit dependencies encode serial/parallel) -``` - -**Properties:** -- Can contain multiple epics as subgraphs -- Dependency edges encode execution order -- `blocks` = serial (bottleneck) -- No dep = parallel (fanout) -- `conditional` = if-fail path - -### The Relationship - -**All epics are mols. Not all mols are epics.** - -| Aspect | Epic | Mol | -|--------|------|-----| -| Shape | Flat (root + children) | Arbitrary DAG | -| Dependencies | Implicit in ordering | Explicit edges | -| Parallelism | Assumed parallel | Encoded in structure | -| Mental model | TODO list | Workflow graph | -| Common use | Simple feature work | Complex orchestration | - -When you `distill` an epic, you get a simple proto. -When you `distill` a complex mol, you get a complex proto (preserving structure). - -## Thermodynamic Properties - -Gas Town's chemistry has thermodynamic properties - work is energy flowing -through the system. - -### Work as Energy - -``` -Proto (potential energy) → Pour/Wisp → Mol/Wisp (kinetic energy) → Squash → Digest (stored work) -``` - -- **Protos** store potential energy - the capability to do work -- **Mols/Wisps** are kinetic - work actively flowing -- **Digests** are stored energy - crystallized outcomes - -### The Audit Trail as Entropy - -Every execution increases entropy - creating more history, more records, -more state. Gas Town manages this through: - -- **Wisps**: High entropy, but evaporates (entropy contained) -- **Squash**: Compresses entropy into minimal digest -- **Distill**: Reduces entropy by extracting reusable pattern - -### The CV Chain - -Every agent has a **chain** of work - their CV: - -``` -Agent CV = ∑(digests) = crystallized capability proof -``` - -Work completed → digest created → agent's chain grows → capability demonstrated. - -This is the foundation for capability-based work matching: your work -history IS your resume. The ledger speaks. - -## The Complete Lifecycle - -### Feature Work (Liquid Path) - -```bash -# 1. Create work from template -bd pour mol-feature --var name=auth - -# 2. Pin to agent -bd pin bd-abc123 --for polecat-ace - -# 3. Agent executes steps -bd update bd-abc123.design --status=in_progress -# ... cognition ... -bd close bd-abc123.design -bd update bd-abc123.implement --status=in_progress -# ... and so on - -# 4. Squash when complete -bd mol squash bd-abc123 --summary="Implemented auth feature" - -# 5. Digest remains in permanent ledger -``` - -### Patrol Work (Vapor Path) - -```bash -# 1. Create wisp (no pin needed) -bd wisp mol-deacon-patrol - -# 2. Execute cycle steps -bd close -bd close -# ... - -# 3. Generate summary and squash -bd mol squash --summary="Patrol complete, no issues" - -# 4. Loop -bd wisp mol-deacon-patrol -# ... -``` - -### Template Creation (Distillation) - -```bash -# 1. Complete some work organically -# ... team develops release workflow over several iterations ... - -# 2. Distill the pattern -bd mol distill bd-release-v3 --as "Release Workflow" - -# 3. Result: new proto available -bd pour mol-release-workflow --var version=2.0 -``` - -## Polymers: Large-Scale Composition - -Protos can compose into arbitrarily large **polymers** - chains of molecules -that encode complex multi-phase work. - -```bash -# Create polymer from multiple protos -bd mol bond mol-design mol-implement --as "Design and Implement" -bd mol bond mol-design-implement mol-test --as "Full Dev Cycle" -bd mol bond mol-full-dev mol-deploy --as "End to End" -``` - -**Polymer properties:** -- Preserve phase relationships from constituent protos -- Can encode hours, days, or weeks of work -- Enable "weekend warrior" autonomous operation -- Beads tracks progress; agents execute; humans sleep - -### The Cognition Sausage Machine - -A large polymer is a **cognition sausage machine**: - -``` -Proto Polymer (input) - │ - ▼ -┌─────────────────────────────────────────┐ -│ GAS TOWN ENGINE │ -│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │ -│ │Pole │ │Pole │ │Pole │ │Pole │ │ -│ │cat │ │cat │ │cat │ │cat │ │ -│ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ │ -│ │ │ │ │ │ -│ └────────┴────────┴────────┘ │ -│ ↓ │ -│ Merge Queue │ -│ ↓ │ -│ Refinery │ -└─────────────────────────────────────────┘ - │ - ▼ -Completed Work + Digests (output) -``` - -Feed in a polymer. Get back completed features, merged PRs, and audit trail. - -## The Proto Library - -Gas Town maintains a **library of curated protos** - the fuel stockpile: - -``` -~/gt/molecules/ -├── mol-shiny/ # Full quality workflow -├── mol-quick-fix/ # Fast path for small changes -├── mol-code-review/ # Pluggable review dimensions -├── mol-release/ # Release workflow -├── mol-deacon-patrol/ # Deacon monitoring cycle -├── mol-witness-patrol/ # Witness worker monitoring -└── mol-polecat-work/ # Standard polecat lifecycle -``` - -**Library operations:** - -```bash -bd mol list # List available protos -bd mol show mol-code-review # Show proto details -bd pour mol-code-review # Instantiate for use -``` - -### Curated vs Organic - -- **Curated protos**: Refined templates in the library, battle-tested -- **Organic protos**: Distilled from real work, may need refinement -- **Path**: Organic → refine → curate → library - -## Digest ID Stability - -When a wisp is created, its head bead ID is **reserved** in permanent storage. -This ensures cross-phase references remain valid: - -``` -1. bd wisp mol-patrol → Creates wisp-123 (ID reserved) -2. bd mol bond ... wisp-123 → Reference created -3. bd mol squash wisp-123 → Digest takes same ID -4. Reference still valid → Points to digest now -``` - -**Implementation:** -- On wisp creation: Write placeholder/tombstone to permanent beads -- On squash: Replace placeholder with actual digest -- Cross-phase references never break - -## Dynamic Bonding: The Christmas Ornament Pattern - -Static molecules have fixed steps defined at design time. But some workflows -need **dynamic structure** - steps that emerge at runtime based on discovered work. - -### The Problem - -Consider mol-witness-patrol. The Witness monitors N polecats where N varies: -- Sometimes 0 polecats (quiet rig) -- Sometimes 8 polecats (busy swarm) -- Polecats come and go during the patrol - -A static molecule can't express "for each polecat, do these steps." - -### The Solution: Dynamic Bond - -The **bond** operator becomes a runtime instantiator: - -```bash -# In survey-workers step: -for polecat in $(gt polecat list gastown); do - bd mol bond mol-polecat-arm $PATROL_WISP_ID \ - --var polecat_name=$polecat \ - --var rig=gastown -done -``` - -Each bond creates a **wisp child** under the patrol molecule: -- `patrol-x7k.arm-ace` (5 steps) -- `patrol-x7k.arm-nux` (5 steps) -- `patrol-x7k.arm-toast` (5 steps) - -### The Christmas Ornament Shape - -``` - ★ mol-witness-patrol (trunk) - /|\ - / | \ - ┌─────────┘ │ └─────────┐ - │ │ │ - PREFLIGHT DISCOVERY CLEANUP - │ │ │ - ┌───┴───┐ ┌───┴───┐ ┌───┴───┐ - │inbox │ │survey │ │aggreg │ - │refnry │ │ │ │save │ - │load │ │ │ │summary│ - └───────┘ └───┬───┘ │contxt │ - │ │loop │ - ┌─────────┼─────────┐ └───────┘ - │ │ │ - ● ● ● mol-polecat-arm - ace nux toast - │ │ │ - ┌──┴──┐ ┌──┴──┐ ┌──┴──┐ - │cap │ │cap │ │cap │ - │ass │ │ass │ │ass │ - │dec │ │dec │ │dec │ - │exec │ │exec │ │exec │ - └──┬──┘ └──┬──┘ └──┬──┘ - │ │ │ - └─────────┴─────────┘ - │ - ⬣ base (cleanup) -``` - -The ornament **hangs from the Witness's pinned bead**. The star is the patrol -head (preflight steps). Arms grow dynamically as polecats are discovered. -The base (cleanup) runs after all arms complete. - -### The WaitsFor Directive - -A step that follows dynamic bonding needs to **wait for all children**: - -```markdown -## Step: aggregate -Collect outcomes from all polecat inspection arms. -WaitsFor: all-children -Needs: survey-workers -``` - -The `WaitsFor: all-children` directive makes this a **fanout gate** - it can't -proceed until ALL dynamically-bonded children complete. - -### Parallelism - -Arms execute in **parallel**. Within an arm, steps are sequential: - -``` -survey-workers ─┬─ arm-ace ─┬─ aggregate - │ (seq) │ - ├─ arm-nux ─┤ (all arms parallel) - │ (seq) │ - └─ arm-toast┘ -``` - -Agents can use subagents (Task tool) to work multiple arms simultaneously. - -### The Activity Feed - -Dynamic bonding enables a **real-time activity feed** - structured work state -instead of agent logs: - -``` -[14:32:01] ✓ patrol-x7k.inbox-check completed -[14:32:03] ✓ patrol-x7k.check-refinery completed -[14:32:07] → patrol-x7k.survey-workers in_progress -[14:32:08] + patrol-x7k.arm-ace bonded (5 steps) -[14:32:08] + patrol-x7k.arm-nux bonded (5 steps) -[14:32:08] + patrol-x7k.arm-toast bonded (5 steps) -[14:32:08] ✓ patrol-x7k.survey-workers completed -[14:32:09] → patrol-x7k.arm-ace.capture in_progress -[14:32:10] ✓ patrol-x7k.arm-ace.capture completed -[14:32:14] ✓ patrol-x7k.arm-ace.decide completed (action: nudge-1) -[14:32:17] ✓ patrol-x7k.arm-ace COMPLETE -[14:32:23] ✓ patrol-x7k SQUASHED → digest-x7k -``` - -This is what you want to see. Not logs. **WORK STATE.** - -The beads ledger becomes a real-time activity feed. Control plane IS data plane. - -### Variable Substitution - -Bonded molecules support variable substitution: - -```markdown -## Molecule: polecat-arm -Inspection cycle for {{polecat_name}} in {{rig}}. - -## Step: capture -Capture tmux output for {{polecat_name}}. -```bash -tmux capture-pane -t gt-{{rig}}-{{polecat_name}} -p | tail -50 -``` - -Variables are resolved at bond time, creating concrete wisp steps. - -### Squash Behavior - -At patrol end: -- **Notable events**: Squash to digest with summary -- **Routine cycle**: Burn without digest - -All arm wisps are children of the patrol wisp - they squash/burn together. - -### The Mol Mall - -mol-polecat-arm is **swappable** via variable: - -```markdown -## Step: survey-workers -For each polecat, bond: {{arm_molecule | default: mol-polecat-arm}} -``` - -Install alternatives from the Mol Mall: -- `mol-polecat-arm-enterprise` (compliance checks) -- `mol-polecat-arm-secure` (credential scanning) -- `mol-polecat-arm-ml` (ML-based stuck detection) - -## Summary of Operators - -| Operator | From | To | Effect | -|----------|------|------|--------| -| `pour` | Proto | Mol | Instantiate as persistent liquid | -| `wisp` | Proto | Wisp | Instantiate as ephemeral vapor | -| `bond` | Any + Any | Compound | Combine (polymorphic) | -| `squash` | Mol/Wisp | Digest | Condense to permanent record | -| `burn` | Wisp | Nothing | Discard without record | -| `distill` | Mol/Epic | Proto | Extract reusable template | -| `pin` | Mol | Agent | Attach work to agent's hook | - -## Design Implications - -### For Beads - -1. **Commands**: `bd pour`, `bd wisp`, `bd pin` -2. **Flags**: `--pour` forces liquid phase when bonding -3. **Wisp storage**: `.beads-wisp/` directory, gitignored -4. **Digest ID reservation**: Placeholder in permanent store on wisp creation - -### For Gas Town - -1. **Daemon**: Don't attach permanent molecules for patrol roles -2. **Deacon template**: Use `bd wisp mol-deacon-patrol` pattern -3. **Polecat lifecycle**: Consider wisp-based with digest on completion -4. **Hook inspection**: `bd hook` command for debugging - -### For Agents - -1. **Polecats**: Receive pinned mols, execute, squash, request shutdown -2. **Patrol roles**: Create wisps, execute cycle, squash, loop -3. **Recovery**: Re-read beads state, continue from last completed step - ---- - -## Appendix: The Vocabulary - -### Lifecycle Phases - -| Term | Meaning | -|------|---------| -| **Rig** | Compose formulas at source level (authoring time) | -| **Cook** | Transform formula to proto (compile time) | -| **Run** | Execute mol/wisp steps (agent execution time) | - -### Artifacts - -| Term | Meaning | -|------|---------| -| **Formula** | Source YAML defining workflow with composition rules | -| **Proto** | Frozen template molecule (solid phase, cooked) | -| **Mol** | Flowing work instance (liquid phase) | -| **Wisp** | Ephemeral execution trace (vapor phase) | -| **Digest** | Condensed permanent record | -| **Polymer** | Large composed proto chain | -| **Epic** | Simple mol shape (flat TODO list) | - -### Operators - -| Term | Meaning | -|------|---------| -| **Pour** | Instantiate proto as mol (solid → liquid) | -| **Wisp** (verb) | Instantiate proto as wisp (solid → vapor) | -| **Bond** | Combine artifacts (polymorphic, symmetric) | -| **Squash** | Condense to digest | -| **Burn** | Discard wisp without digest | -| **Distill** | Extract proto from experience | - -### Agent Mechanics - -| Term | Meaning | -|------|---------| -| **Hook** | Agent's attachment point for work | -| **Pin** | Attach mol to agent's hook | -| **Sling** | Cook + assign to agent hook | - ---- - -*The chemistry is the interface. The ledger is the truth. The work gets done.* diff --git a/docs/molecule-algebra.md b/docs/molecule-algebra.md deleted file mode 100644 index 7f80e30f..00000000 --- a/docs/molecule-algebra.md +++ /dev/null @@ -1,729 +0,0 @@ -# Molecule Algebra: A Composition Language for Work (MEOW) - -> Status: Design Spec v1 - December 2024 -> -> "From 'issues in git' to a work composition algebra in 10 weeks." - -## Overview - -This document defines **MEOW** - the **M**olecular **E**xpression **O**f **W**ork - -a declarative algebra for composing, transforming, and executing structured work. -The algebra enables mechanical composition of workflows without AI, reserving -cognition for leaf-node execution only. - -**Key insight**: Structure is computation. Content is cognition. They're separate. - -``` -Molecules = Graph Algebra (mechanical, gt executes) -Steps = AI Cognition (agent provides) -``` - -## The Three Phases: Rig, Cook, Run - -Gas Town work flows through three phases: - -``` -RIG ────→ COOK ────→ RUN -``` - -| Phase | What Happens | Operator | Output | -|-------|--------------|----------|--------| -| **Rig** | Compose formulas (source level) | extends, compose | Compound Formula | -| **Cook** | Instantiate work | cook, pour, wisp | Proto, Mol, Wisp | -| **Run** | Execute steps | (agent execution) | Work done | - -See [molecular-chemistry.md](molecular-chemistry.md) for the full specification. - -## Formulas and Cooking - -**Formulas** are the source code; **rigging** composes them; **cooking** -produces executable artifacts. - -### The Artifact Tiers - -``` -Formula (.formula.yaml) ← Source code - ↓ rig (compose) ← Source-level composition -Compound Formula ← Combined source - ↓ cook ← Pre-expand, flatten -Proto (frozen in beads) ← Compiled, flat graph - ↓ pour/wisp ← Instantiate -Mol/Wisp (running) ← The work flowing -``` - -| Tier | Name | Format | Nature | -|------|------|--------|--------| -| Source | **Formula** | YAML | Composable via `extends`/`compose` | -| Compiled | **Proto** | Beads issue | Frozen, flat graph, fast instantiation | -| Running | **Mol/Wisp** | Beads issue | Active, flowing work | - -### Two Composition Operators - -| Operator | Level | Inputs | Output | -|----------|-------|--------|--------| -| **Rig** | Source | Formula + Formula | Compound Formula | -| **Bond** | Artifact | Proto/Mol/Wisp + any | Combined artifact | - -**Rig** is source-level composition (formula YAML with `extends`, `compose`). -**Bond** is artifact-level composition (combining cooked protos, linking mols). - -See the Bond Table in [molecular-chemistry.md](molecular-chemistry.md) for full semantics. - -### Why Cook? - -Cooking **pre-expands** all composition at "compile time": -- Macros (Rule of Five) expand to flat steps -- Aspects apply to matching pointcuts -- Branches and gates wire up -- Result: pure step graph with no interpretation needed - -Instantiation then becomes pure **copy + variable substitution**. Fast, mechanical, -deterministic. - -### Formula Format - -Formulas are YAML files with `.formula.yaml` extension. YAML for human -readability (humans author these; agents cook them): - -```yaml -formula: shiny -description: Engineer in a Box - the canonical right way -version: 1 -steps: - - id: design - description: Think carefully about architecture - - id: implement - needs: [design] - - id: review - needs: [implement] - - id: test - needs: [review] - - id: submit - needs: [test] -``` - -Formulas with composition: - -```yaml -formula: shiny-enterprise -extends: shiny -version: 1 -compose: - - expand: - target: implement - with: rule-of-five - - aspect: - pointcut: "implement.*" - with: security-audit - - gate: - before: submit - condition: "security-postscan.approved" -``` - -### CLI - -```bash -# Cook a formula into a proto -bd cook shiny-enterprise -# "Cooking shiny-enterprise..." -# "✓ Cooked proto: shiny-enterprise (30 steps)" - -# Preview without saving -bd cook shiny-enterprise --dry-run - -# List available formulas -bd formula list - -# Show formula details -bd formula show rule-of-five - -# Instantiate the cooked proto -bd pour shiny-enterprise --var feature="auth" -``` - -### Formula Storage - -``` -~/.beads/formulas/ # User formulas -~/gt/.beads/formulas/ # Town formulas -.beads/formulas/ # Project formulas -``` - -## The Phases of Matter - -Work in Gas Town exists in three phases: - -| Phase | Name | Storage | Lifecycle | Use Case | -|-------|------|---------|-----------|----------| -| **Solid** | Proto | `.beads/` (template) | Frozen, reusable | Workflow patterns | -| **Liquid** | Mol | `.beads/` | Flowing, persistent | Project work, audit trail | -| **Vapor** | Wisp | `.beads-wisp/` | Ephemeral, evaporates | Execution scaffolding | - -### Phase Transitions - -``` - ┌─────────────────┐ - │ PROTO │ - │ (solid) │ - └────────┬────────┘ - │ - ┌──────────────┼──────────────┐ - │ │ │ - pour wisp bond - │ │ │ - ▼ ▼ ▼ - ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ - │ MOL │ │ WISP │ │ COMPOUND │ - │ (liquid) │ │ (vapor) │ │ PROTO │ - └───────┬───────┘ └───────┬───────┘ └───────────────┘ - │ │ - squash squash/burn - │ │ - ▼ ▼ - ┌───────────────┐ ┌───────────────┐ - │ DIGEST │ │ (evaporates) │ - │ (condensed) │ │ or digest │ - └───────────────┘ └───────────────┘ -``` - -### Phase Verbs - -| Verb | Transition | Effect | -|------|------------|--------| -| `pour` | solid → liquid | Create persistent mol | -| `wisp` | solid → vapor | Create ephemeral wisp | -| `bond` | any + any | Combine (polymorphic) | -| `squash` | liquid/vapor → digest | Condense to record | -| `burn` | vapor → nothing | Discard without record | - -## The Mol/Wisp Decision - -**"Is this the work, or is this wrapping the work?"** - -| Pour when... | Wisp when... | -|---------------------|----------------------| -| This IS the work item | This SHAPES execution | -| Multiple agents coordinate | Single agent executes | -| Stakeholders track progress | Only outcome matters | -| Cross-rig visibility needed | Local execution detail | -| CV/audit trail value | Scaffolding, process | - -The `--on` flag implies wisp: `gt sling shiny gastown/Toast --on gt-abc123` - -## Graph Primitives - -### Steps - -A step is a unit of work with: -- `id`: Unique identifier within molecule -- `description`: What to do (consumed by agent) -- `needs`: Dependencies (steps that must complete first) -- `output`: Structured result (available to conditions) - -```yaml -- id: implement - description: Write the authentication module - needs: [design] -``` - -### Edges - -Dependencies between steps: -- `needs`: Hard dependency (must complete first) -- `blocks`: Inverse of needs (this step blocks that one) -- `conditional`: Only if condition met - -### Molecules - -A molecule is a DAG of steps: - -```yaml -molecule: shiny -description: Engineer in a Box - the canonical right way - -steps: - - id: design - description: Think carefully about architecture - - - id: implement - description: Write the code - needs: [design] - - - id: review - description: Code review - needs: [implement] - - - id: test - description: Run tests - needs: [review] - - - id: submit - description: Submit for merge - needs: [test] -``` - -## Composition Operators - -### Sequential Composition - -```yaml -# A then B -sequence: - - step-a - - step-b -``` - -Or implicitly via `needs`: -```yaml -- id: b - needs: [a] -``` - -### Parallel Composition - -Steps without dependencies run in parallel: -```yaml -- id: unit-tests - needs: [implement] - -- id: integration-tests - needs: [implement] - -- id: review - needs: [unit-tests, integration-tests] # Waits for both -``` - -### Branching - -Add parallel paths that rejoin: - -```yaml -compose: - - branch: - from: implement - steps: [perf-test, load-test, chaos-test] - join: review -``` - -Produces: -``` -implement ─┬─ perf-test ──┬─ review - ├─ load-test ──┤ - └─ chaos-test ─┘ -``` - -### Looping - -Fixed iteration: -```yaml -compose: - - loop: - count: 5 - body: [refine] -``` - -Conditional iteration: -```yaml -compose: - - loop: - step: review - until: "review.output.approved == true" - max: 3 # Safety bound -``` - -### Gates - -Wait for condition before proceeding: -```yaml -compose: - - gate: - before: submit - condition: "security-scan.output.passed == true" -``` - -## Advice Operators (Lisp-style!) - -Inspired by Lisp advice and AOP, these operators inject behavior -without modifying the original molecule. - -### Before - -Insert step before target: -```yaml -compose: - - advice: - target: review - before: security-scan -``` - -### After - -Insert step after target: -```yaml -compose: - - advice: - target: implement - after: run-linter -``` - -### Around - -Wrap target with before/after: -```yaml -compose: - - advice: - target: "*.implement" - around: - before: log-start - after: log-end -``` - -### Pattern Matching - -Target supports glob patterns: -```yaml -# All implement steps in any molecule -target: "*.implement" - -# All steps in shiny -target: "shiny.*" - -# Specific step -target: "shiny.review" - -# All steps (wildcard) -target: "*" -``` - -## Expansion Operators (Macros!) - -Expansion operators transform structure at bond time. - -### Simple Expansion - -Apply a template to a target step: -```yaml -compose: - - expand: - target: implement - with: rule-of-five -``` - -The `rule-of-five` template: -```yaml -molecule: rule-of-five -type: expansion -description: Jeffrey's Rule - iterate 4-5 times for convergence - -template: - - id: "{target}.draft" - description: "Initial attempt: {target.description}" - - - id: "{target}.refine-1" - description: "Refine for correctness" - needs: ["{target}.draft"] - - - id: "{target}.refine-2" - description: "Refine for clarity" - needs: ["{target}.refine-1"] - - - id: "{target}.refine-3" - description: "Refine for edge cases" - needs: ["{target}.refine-2"] - - - id: "{target}.refine-4" - description: "Final polish" - needs: ["{target}.refine-3"] -``` - -Result: `implement` becomes 5 steps with proper dependency wiring. - -### Map Expansion - -Apply template to all matching steps: -```yaml -compose: - - map: - select: "shiny.*" - with: rule-of-five -``` - -All 5 shiny steps get R5 treatment → 25 total steps. - -## Aspects (AOP) - -Cross-cutting concerns applied to multiple join points: - -```yaml -aspect: security-audit -description: Security scanning at implementation boundaries - -pointcuts: - - glob("*.implement") - - glob("*.submit") - -advice: - around: - before: - - step: security-prescan - args: { target: "{step.id}" } - after: - - step: security-postscan - args: { target: "{step.id}" } - - gate: - condition: "security-postscan.output.approved == true" -``` - -Apply aspects at bond time: -```bash -bd bond shiny --with-aspect security-audit --with-aspect logging -``` - -## Selection Operators - -For targeting steps in advice/expansion: - -| Selector | Matches | -|----------|---------| -| `step("review")` | Specific step by ID | -| `glob("*.implement")` | Pattern match | -| `glob("shiny.*")` | All steps in molecule | -| `filter(status == "open")` | Predicate match | -| `children(step)` | Direct children | -| `descendants(step)` | All descendants | - -## Conditions - -Conditions are evaluated mechanically (no AI): - -```yaml -# Step status -"step.status == 'complete'" - -# Step output (structured) -"step.output.approved == true" -"step.output.errors.count == 0" - -# Aggregates -"steps.complete >= 3" -"children(step).all(status == 'complete')" - -# External checks -"file.exists('go.mod')" -"env.CI == 'true'" -``` - -Conditions are intentionally limited to keep evaluation decidable. - -## Runtime Dynamic Expansion - -For discovered work at runtime (Christmas Ornament pattern): - -```yaml -step: survey-workers -on-complete: - for-each: output.discovered_workers - bond: polecat-arm - with-vars: - polecat: "{item.name}" - rig: "{item.rig}" -``` - -The `for-each` evaluates against step output, bonding N instances dynamically. -Still declarative, still mechanical. - -## Polymorphic Bond Operator - -`bond` combines molecules with context-aware phase behavior: - -| Operands | Result | -|----------|--------| -| proto + proto | compound proto (frozen) | -| proto + mol | pour proto, attach | -| proto + wisp | wisp proto, attach | -| mol + mol | link via edges | -| wisp + wisp | link via edges | -| mol + wisp | reference link (cross-phase) | -| expansion + workflow | expanded proto (macro) | -| aspect + molecule | advised molecule | - -Phase override flags: -- `--pour`: Force creation as mol -- `--wisp`: Force creation as wisp - -## Complete Example: Shiny-Enterprise - -```yaml -molecule: shiny-enterprise -extends: shiny -description: Full enterprise engineering workflow - -compose: - # Apply Rule of Five to implement step - - expand: - target: implement - with: rule-of-five - - # Security aspect on all implementation steps - - aspect: - pointcut: "implement.*" - with: security-audit - - # Gate on security approval before submit - - gate: - before: submit - condition: "security-postscan.approved == true" - - # Parallel performance testing branch - - branch: - from: implement.refine-4 - steps: [perf-test, load-test, chaos-test] - join: review - - # Loop review until approved (max 3 attempts) - - loop: - step: review - until: "review.output.approved == true" - max: 3 - - # Logging on all steps - - advice: - target: "*" - before: log-start - after: log-end -``` - -gt compiles this to ~30+ steps with proper dependencies. -Agent executes. AI provides cognition for each step. -Structure is pure algebra. - -## The Grammar - -``` -MOLECULE ::= 'molecule:' ID steps compose? - -STEPS ::= step+ -STEP ::= 'id:' ID 'description:' TEXT needs? -NEEDS ::= 'needs:' '[' ID+ ']' - -COMPOSE ::= 'compose:' rule+ -RULE ::= advice | insert | branch | loop | gate | expand | aspect - -ADVICE ::= 'advice:' target before? after? around? -TARGET ::= 'target:' PATTERN -BEFORE ::= 'before:' STEP_REF -AFTER ::= 'after:' STEP_REF -AROUND ::= 'around:' '{' before? after? '}' - -BRANCH ::= 'branch:' from steps join -LOOP ::= 'loop:' (count | until) body max? -GATE ::= 'gate:' before? condition - -EXPAND ::= 'expand:' target 'with:' TEMPLATE -MAP ::= 'map:' select 'with:' TEMPLATE - -ASPECT ::= 'aspect:' ID pointcuts advice -POINTCUTS ::= 'pointcuts:' selector+ -SELECTOR ::= glob | filter | step - -CONDITION ::= field OP value | aggregate OP value | external -FIELD ::= step '.' attr | 'output' '.' path -AGGREGATE ::= 'children' | 'descendants' | 'steps' '.' stat -EXTERNAL ::= 'file.exists' | 'env.' key -``` - -## Decidability - -The algebra is intentionally restricted: -- Loops have max iteration bounds -- Conditions limited to step/output inspection -- No recursion in expansion templates -- No arbitrary code execution - -This keeps evaluation decidable and safe for mechanical execution. - -## Safety Constraints - -The cooker enforces these constraints to prevent runaway expansion: - -### Cycle Detection -Circular `extends` chains are detected and rejected: -``` -A extends B extends C extends A → ERROR: cycle detected -``` - -### Aspect Self-Matching Prevention -Aspects only match *original* steps, not steps inserted by the same aspect. -Without this, a pointcut like `*.implement` that inserts `security-prescan` -could match its own insertion infinitely. - -### Maximum Expansion Depth -Nested expansions are bounded (default: 5 levels). This allows massive work -generation while preventing runaway recursion. Configurable via: -```yaml -cooking: - max_expansion_depth: 5 -``` - -### Graceful Degradation -Cooking errors produce warnings, not failures where possible. Philosophy: -get it working as well as possible, warn the human, continue. Invalid steps -may be annotated with error metadata rather than blocking the entire cook. - -Errors are written to the Gas Town escalation channel for human review. - -## What This Enables - -1. **Composition without AI**: gt compiles molecule algebra mechanically -2. **Marketplace of primitives**: Aspects, wrappers, expansions as tradeable units -3. **Deterministic expansion**: Same input → same graph, always -4. **AI for content only**: Agents execute steps, don't construct structure -5. **Inspection/debugging**: See full expanded graph before execution -6. **Optimization**: gt can parallelize, dedupe, optimize the graph -7. **Roles/Companies in a box**: Compose arbitrary organizational workflows - -## The Vision - -``` - ┌─────────────────────────────────────────┐ - │ MOL MALL (Marketplace) │ - │ ┌─────────┐ ┌─────────┐ ┌───────────┐ │ - │ │ Shiny │ │ Rule of │ │ Security │ │ - │ │ Formula │ │ Five │ │ Aspect │ │ - │ └─────────┘ └─────────┘ └───────────┘ │ - │ ┌─────────┐ ┌─────────┐ ┌───────────┐ │ - │ │Planning │ │ Release │ │ Company │ │ - │ │ Formula │ │ Formula │ │ Onboard │ │ - │ └─────────┘ └─────────┘ └───────────┘ │ - └─────────────────────────────────────────┘ - │ - ▼ compose - ┌─────────────────────────────────────────┐ - │ YOUR ORGANIZATION FORMULA │ - │ │ - │ Planning + Shiny + R5 + Security + │ - │ Release + Onboarding = Company in Box │ - │ │ - └─────────────────────────────────────────┘ - │ - ▼ cook - ┌─────────────────────────────────────────┐ - │ PURE PROTO │ - │ Pre-expanded, flat graph │ - │ No macros, no aspects, just steps │ - └─────────────────────────────────────────┘ - │ - ▼ pour/wisp - ┌─────────────────────────────────────────┐ - │ GAS TOWN │ - │ Polecats execute. Wisps evaporate. │ - │ Mols persist. Digests accumulate. │ - │ Work gets done. │ - └─────────────────────────────────────────┘ -``` - -From issues in git → work composition algebra → companies in a box. - ---- - -*Structure is computation. Content is cognition. The work gets done.* diff --git a/docs/molecules.md b/docs/molecules.md deleted file mode 100644 index 968dfac3..00000000 --- a/docs/molecules.md +++ /dev/null @@ -1,346 +0,0 @@ -# Molecules: Composable Workflow Templates - -This document covers the molecule system in depth. - -For an overview, see [architecture.md](architecture.md#molecules-composable-workflow-templates). -For the full lifecycle (Rig → Cook → Run), see [molecular-chemistry.md](molecular-chemistry.md). - -## Core Concepts - -A **molecule** is a workflow template stored as a beads issue with `labels: ["template"]`. -When bonded, it creates child issues forming a DAG of steps. - -``` -Proto (template) - │ - ▼ bd mol bond -┌─────────────────┐ -│ Mol (durable) │ ← .beads/ (synced, auditable) -│ or │ -│ Wisp (ephemeral)│ ← .beads-wisp/ (gitignored) -└────────┬────────┘ - │ - ┌─────┴─────┐ - ▼ ▼ -bd mol burn bd mol squash -(no record) (creates digest) -``` - -| Concept | Description | -|---------|-------------| -| Proto | Template molecule (is_template: true in beads) | -| Mol | Durable instance in .beads/ | -| Wisp | Ephemeral instance in .beads-wisp/ | -| Digest | Condensed completion record | -| Bond | Instantiate proto → mol/wisp | - -## Molecule Storage - -Molecules are standard beads issues stored in `molecules.jsonl`: - -```json -{ - "id": "mol-shiny", - "title": "Shiny: {{feature_name}}", - "description": "Full workflow from design to merge.\n\nVars:\n- {{feature_name}} - What to build", - "labels": ["template"], - "issue_type": "epic" -} -``` - -**Loaded from multiple sources** (later overrides earlier): -1. Built-in (embedded in bd binary) -2. Town-level: `~/gt/.beads/molecules.jsonl` -3. User-level: `~/.beads/molecules.jsonl` -4. Project-level: `.beads/molecules.jsonl` - -## Variable Substitution - -Molecules support `{{var}}` placeholders resolved at bond time: - -```bash -bd mol bond mol-shiny --var feature_name="user auth" -# Creates: "Shiny: user auth" -``` - -Variables work in: -- Title -- Description -- Child step titles/descriptions - -## Steps as Children - -Steps are hierarchical children of the molecule: - -``` -mol-shiny (epic) -├── mol-shiny.1 "Design" -├── mol-shiny.2 "Implement" Needs: .1 -├── mol-shiny.3 "Review" Needs: .2 -├── mol-shiny.4 "Test" Needs: .3 -└── mol-shiny.5 "Submit" Needs: .4 -``` - -Dependencies are encoded in beads edges, not in step descriptions. - -## Molecule CLI Reference - -### bd mol bond - -Instantiate a proto molecule into a runnable mol or wisp. - -```bash -bd mol bond [--wisp] [--var key=value...] -``` - -| Flag | Description | -|------|-------------| -| `--wisp` | Create ephemeral wisp instead of durable mol | -| `--var` | Variable substitution (repeatable) | -| `--ref` | Custom reference ID for the instance | - -**Examples:** - -```bash -# Durable mol -bd mol bond mol-shiny --var feature_name="auth" - -# Ephemeral wisp for patrol -bd mol bond mol-witness-patrol --wisp - -# Dynamic child with custom ref (Christmas Ornament pattern) -bd mol bond mol-polecat-arm $PATROL_ID --ref arm-ace --var polecat=ace -``` - -### bd mol squash - -Complete a molecule and generate a permanent digest. - -```bash -bd mol squash --summary='...' -``` - -The summary is agent-generated - the intelligence comes from the agent, not beads. - -**For Mol**: Creates digest in .beads/, original steps remain. -**For Wisp**: Evaporates wisp, creates digest. Execution trace gone, outcome preserved. - -### bd mol burn - -Abandon a molecule without record. - -```bash -bd mol burn [--reason='...'] -``` - -Use burn for: -- Routine patrol cycles (no audit needed) -- Failed experiments -- Cancelled work - -Use squash for: -- Completed work -- Investigations with findings -- Anything worth recording - -### bd mol list - -List available molecules: - -```bash -bd mol list # All templates -bd mol list --label plugin # Plugin molecules only -``` - -## Plugins ARE Molecules - -Patrol plugins are molecules with specific labels: - -```json -{ - "id": "mol-security-scan", - "title": "Security scan for {{polecat_name}}", - "description": "Check for vulnerabilities.\n\nVars: {{polecat_name}}, {{captured_output}}", - "labels": ["template", "plugin", "witness", "tier:haiku"], - "issue_type": "task" -} -``` - -Label conventions: -- `plugin` - marks as bondable at hook points -- `witness` / `deacon` / `refinery` - which patrol uses it -- `tier:haiku` / `tier:sonnet` - model hint - -**Execution in patrol:** - -```bash -# plugin-run step bonds registered plugins: -bd mol bond mol-security-scan $PATROL_WISP \ - --ref security-{{polecat_name}} \ - --var polecat_name=ace \ - --var captured_output="$OUTPUT" -``` - -## The Christmas Ornament Pattern - -Dynamic bonding creates tree structures at runtime: - -``` - ★ mol-witness-patrol - /|\ - ┌─────┘ │ └─────┐ - PREFLIGHT │ CLEANUP - │ │ │ - ┌───┴───┐ │ ┌───┴───┐ - │inbox │ │ │aggreg │ - │load │ │ │summary│ - └───────┘ │ └───────┘ - │ - ┌─────────┼─────────┐ - │ │ │ - ● ● ● mol-polecat-arm (dynamic) - ace nux toast - │ │ │ - ┌──┴──┐ ┌──┴──┐ ┌──┴──┐ - │steps│ │steps│ │steps│ - └──┬──┘ └──┬──┘ └──┬──┘ - │ │ │ - └─────────┴─────────┘ - │ - ⬣ base -``` - -**Key primitives:** -- `bd mol bond ... --ref` - creates named children -- `WaitsFor: all-children` - fanout gate -- Arms execute in parallel - -## Mol Mall - -Distribution through molecule marketplace: - -```bash -# Install from registry -bd mol install mol-security-scan - -# Updates ~/.beads/molecules.jsonl -``` - -Mol Mall serves `molecules.jsonl` fragments. Installation appends to your catalog. - -## Code Review Molecule - -The code-review molecule is a pluggable workflow: - -``` -mol-code-review -├── discovery (parallel) -│ ├── file-census -│ ├── dep-graph -│ └── coverage-map -│ -├── structural (sequential) -│ ├── architecture-review -│ └── abstraction-analysis -│ -├── tactical (parallel per component) -│ ├── security-scan -│ ├── performance-review -│ └── complexity-analysis -│ -└── synthesis (single) - └── aggregate -``` - -Each dimension is a molecule that can be: -- Swapped for alternatives from Mol Mall -- Customized by forking and installing your version -- Disabled by not bonding it - -**Usage:** - -```bash -bd mol bond mol-code-review --var scope="src/" -``` - -## Lifecycle Summary - -Gas Town work follows the **Rig → Cook → Run** lifecycle: - -``` - RIG (source) COOK (artifact) RUN (execution) - ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ - │ Formula │──────►│ Proto │──────►│ Mol/Wisp │ - │ (.yaml) │ cook │ (cooked/frozen)│ pour/ │ (flowing) │ - │ │ │ │ wisp │ │ - └─────────────────┘ └─────────────────┘ └────────┬────────┘ - │ - ┌────┴────┐ - ▼ ▼ - burn squash - │ │ - ▼ ▼ - (gone) Digest - (permanent) -``` - -**Full lifecycle:** -- **Formula** (.formula.toml or .formula.json) = Recipe (source code for workflows) -- **Proto** = Fuel (cooked template, ready to instantiate) -- **Mol/Wisp** = Steam (active execution) -- **Digest** = Distillate (crystallized work) - -See [molecular-chemistry.md](molecular-chemistry.md) for the complete specification. - -## Formula File Formats - -Formulas support both TOML (preferred) and JSON formats: - -```bash -# List available formulas -bd formula list - -# Show formula details -bd formula show shiny - -# Convert JSON to TOML (better for human editing) -bd formula convert shiny # Single formula -bd formula convert --all # All formulas -bd formula convert shiny --stdout # Preview -``` - -**Why TOML?** -- Multi-line strings without `\n` escaping -- Comments allowed -- Human-readable diffs -- Same parsing semantics as JSON - -The loader tries `.formula.toml` first, falling back to `.formula.json`. - -### Example Formula (TOML) - -```toml -formula = "shiny" -type = "workflow" -version = 1 -description = """ -Engineer in a Box - design before code, review before ship. -""" - -[vars.feature] -description = "The feature being implemented" -required = true - -[[steps]] -id = "design" -title = "Design {{feature}}" -description = """ -Think carefully about architecture before writing code. -Consider edge cases and simpler approaches. -""" - -[[steps]] -id = "implement" -title = "Implement {{feature}}" -needs = ["design"] -``` diff --git a/docs/no-tmux-mode.md b/docs/no-tmux-mode.md deleted file mode 100644 index f3d8ab25..00000000 --- a/docs/no-tmux-mode.md +++ /dev/null @@ -1,122 +0,0 @@ -# No-Tmux Mode - -Gas Town can operate without tmux or the daemon, using beads as the universal data plane for passing args and context. - -## Background - -Tmux instability can crash workers. The daemon relies on tmux for: -- Session management (creating/killing panes) -- Nudging agents via SendKeys -- Crash detection via pane-died hooks - -When tmux is unstable, the entire operation fails. No-tmux mode enables continued operation in degraded mode. - -## Key Insight: Beads Replace SendKeys - -In normal mode, `--args` are injected via tmux SendKeys. In no-tmux mode: -- Args are stored in the pinned bead description (`attached_args` field) -- `gt prime` reads and displays args from the pinned bead -- No prompt injection needed - agents discover everything via `bd show` - -## Usage - -### Slinging Work with Args - -```bash -# Normal mode: args injected via tmux + stored in bead -gt sling gt-abc --args "patch release" - -# In no-tmux mode: nudge fails gracefully, but args are in the bead -# Agent discovers args via gt prime when it starts -``` - -### Spawning without Tmux - -```bash -# Use --naked to skip tmux session creation -gt sling gt-abc gastown --naked - -# Output tells you how to start the agent manually: -# cd ~/gt/gastown/polecats/ -# claude -``` - -### Agent Discovery - -When an agent starts (manually or via IDE), the SessionStart hook runs `gt prime`, which: -1. Detects the agent's role from cwd -2. Finds pinned work -3. Displays attached args prominently -4. Shows current molecule step - -The agent sees: - -``` -## ATTACHED WORK DETECTED - -Pinned bead: gt-abc -Attached molecule: gt-xyz -Attached at: 2025-12-26T12:00:00Z - -ARGS (use these to guide execution): - patch release - -**Progress:** 0/5 steps complete -``` - -## What Works vs What's Degraded - -### What Still Works - -| Feature | How It Works | -|---------|--------------| -| Propulsion via pinned beads | Agents pick up work on startup | -| Self-handoff | Agents can cycle themselves | -| Patrol loops | Deacon, Witness, Refinery keep running | -| Mail system | Beads-based, no tmux needed | -| Args passing | Stored in bead description | -| Work discovery | `gt prime` reads from bead | - -### What Is Degraded - -| Limitation | Impact | -|------------|--------| -| No interrupts | Cannot nudge busy agents mid-task | -| Polling only | Agents must actively check inbox (no push) | -| Await steps block | "Wait for human" steps require manual agent restart | -| No crash detection | pane-died hooks unavailable | -| Manual startup | Human must start each agent in separate terminal | - -### Workflow Implications - -- **Patrol agents** work fine (they poll as part of their loop) -- **Task workers** need restart to pick up new work -- Cannot redirect a busy worker to urgent task -- Human must monitor and restart crashed agents - -## Commands Summary - -| Command | Purpose | -|---------|---------| -| `gt sling --args "..."` | Store args in bead, nudge gracefully | -| `gt sling --naked` | Assign work without tmux session | -| `gt prime` | Display attached work + args on startup | -| `gt mol status` | Show current work status including args | -| `bd show ` | View raw bead with attached_args field | - -## Implementation Details - -Args are stored in the bead description as a `key: value` field: - -``` -attached_molecule: gt-xyz -attached_at: 2025-12-26T12:00:00Z -attached_args: patch release -``` - -The `beads.AttachmentFields` struct includes: -- `AttachedMolecule` - the work molecule ID -- `AttachedAt` - timestamp when attached -- `AttachedArgs` - natural language instructions - -These are parsed by `beads.ParseAttachmentFields()` and formatted by `beads.FormatAttachmentFields()`. diff --git a/docs/pinned-beads-design.md b/docs/pinned-beads-design.md deleted file mode 100644 index dc02f2cf..00000000 --- a/docs/pinned-beads-design.md +++ /dev/null @@ -1,652 +0,0 @@ -# Pinned Beads Architecture - -> **Status**: Design Draft -> **Author**: max (crew) -> **Date**: 2025-12-23 - -## Overview - -Every Gas Town agent has a **pinned bead** - a persistent hook that serves as their -work attachment point. This document formalizes the semantics, discovery mechanism, -and lifecycle of pinned beads across all agent roles. - -## The Pinned Bead Concept - -A pinned bead is: -- A bead with `status: pinned` (never closes) -- Titled `"{role} Handoff"` (e.g., "Toast Handoff" for polecat Toast) -- Contains attachment fields in its description when work is slung - -``` -┌─────────────────────────────────────────────┐ -│ Pinned Bead: "Toast Handoff" │ -│ Status: pinned │ -│ Description: │ -│ attached_molecule: gt-xyz │ -│ attached_at: 2025-12-23T15:30:45Z │ -└─────────────────────────────────────────────┘ - │ - ▼ (points to) -┌─────────────────────────────────────────────┐ -│ Molecule: gt-xyz │ -│ Title: "Implement feature X" │ -│ Type: epic (molecule root) │ -│ Children: gt-abc, gt-def, gt-ghi (steps) │ -└─────────────────────────────────────────────┘ -``` - -## Role-by-Role Pinned Bead Semantics - -### 1. Polecat - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `{name} Handoff` (e.g., "Toast Handoff") | -| **Beads Location** | Rig-level (`.beads/` in rig root) | -| **Prefix** | `gt-*` | -| **Created By** | `gt sling` on first assignment | -| **Typical Content** | Molecule for feature/bugfix work | -| **Lifecycle** | Created → Work attached → Work detached → Dormant | - -**Discovery**: -```bash -gt mol status # Shows what's on your hook -bd list --status=pinned # Low-level: finds pinned beads -``` - -**Protocol**: -1. On startup, check `gt mol status` -2. If work attached → Execute molecule steps -3. If no work → Check mail inbox for new assignments -4. On completion → Work auto-detached, check again - -### 2. Crew - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `{name} Handoff` (e.g., "joe Handoff") | -| **Beads Location** | Clone-level (`.beads/` in crew member's clone) | -| **Prefix** | `gt-*` | -| **Created By** | `gt sling` or manual pinned bead creation | -| **Typical Content** | Longer-lived work, multi-session tasks | -| **Lifecycle** | Persistent across sessions, human-managed | - -**Key difference from Polecat**: -- No witness monitoring -- Human decides when to detach work -- Work persists until explicitly completed - -**Discovery**: Same as polecat (`gt mol status`) - -### 3. Witness - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `Witness Handoff` | -| **Beads Location** | Rig-level | -| **Prefix** | `gt-*` | -| **Created By** | Deacon or Mayor sling | -| **Typical Content** | Patrol wisp (ephemeral) | -| **Lifecycle** | Wisp attached → Patrol executes → Wisp squashed → New wisp | - -**Protocol**: -1. On startup, check `gt mol status` -2. If wisp attached → Execute patrol cycle -3. Squash wisp on completion (creates digest) -4. Loop or await new sling - -### 4. Refinery - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `Refinery Handoff` | -| **Beads Location** | Rig-level | -| **Prefix** | `gt-*` | -| **Created By** | Witness or Mayor sling | -| **Typical Content** | Epic with batch of issues | -| **Lifecycle** | Epic attached → Dispatch to polecats → Monitor → Epic completed | - -**Protocol**: -1. Check `gt mol status` for attached epic -2. Dispatch issues to polecats via `gt sling issue polecat/name` -3. Monitor polecat progress -4. Report completion when all issues closed - -### 5. Deacon - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `Deacon Handoff` | -| **Beads Location** | Town-level (`~/gt/.beads/`) | -| **Prefix** | `hq-*` | -| **Created By** | Mayor sling or self-loop | -| **Typical Content** | Patrol wisp (always ephemeral) | -| **Lifecycle** | Wisp → Execute → Squash → Loop | - -**Protocol**: -1. Check `gt mol status` -2. Execute patrol wisp steps -3. Squash wisp to digest -4. Self-sling new patrol wisp and loop - -### 6. Mayor - -| Aspect | Value | -|--------|-------| -| **Title Pattern** | `Mayor Handoff` | -| **Beads Location** | Town-level (`~/gt/.beads/`) | -| **Prefix** | `hq-*` | -| **Created By** | External sling or self-assignment | -| **Typical Content** | Strategic work, cross-rig coordination | -| **Lifecycle** | Human-managed like Crew | - -**Key difference**: Mayor is human-controlled (like Crew), but operates at -town level with visibility into all rigs. - -## Summary Table - -| Role | Title Pattern | Beads Level | Prefix | Ephemeral? | Managed By | -|------|---------------|-------------|--------|------------|------------| -| Polecat | `{name} Handoff` | Rig | `gt-` | No | Witness | -| Crew | `{name} Handoff` | Clone | `gt-` | No | Human | -| Witness | `Witness Handoff` | Rig | `gt-` | Yes (wisp) | Deacon | -| Refinery | `Refinery Handoff` | Rig | `gt-` | No | Witness | -| Deacon | `Deacon Handoff` | Town | `hq-` | Yes (wisp) | Self/Mayor | -| Mayor | `Mayor Handoff` | Town | `hq-` | No | Human | - ---- - -## Discovery Mechanism - -### How Does a Worker Find Its Pinned Bead? - -Workers find their pinned bead through a **title-based lookup**: - -```go -// In beads.go -func (b *Beads) FindHandoffBead(role string) (*Issue, error) { - issues := b.List(ListOptions{Status: StatusPinned}) - targetTitle := HandoffBeadTitle(role) // "{role} Handoff" - for _, issue := range issues { - if issue.Title == targetTitle { - return issue, nil - } - } - return nil, nil // Not found -} -``` - -**CLI path**: -```bash -gt mol status [target] -``` - -This: -1. Determines the agent's role/identity from `[target]` or environment -2. Calls `FindHandoffBead(role)` to find the pinned bead -3. Parses `AttachmentFields` from the description -4. Shows attached molecule and progress - -### Current Limitation: Role Naming - -The current implementation uses simple role names: -- Polecat: Uses polecat name (e.g., "Toast") -- Others: Use role name (e.g., "Witness", "Refinery") - -**Problem**: If there are multiple polecats, each needs a unique handoff bead. - -**Solution (current)**: The role passed to `FindHandoffBead` is the polecat's -name, not "polecat". So "Toast Handoff" is different from "Alpha Handoff". - ---- - -## Dashboard Visibility - -### How Do Users See Pinned Beads? - -**Current state**: Limited visibility - -**Proposed commands**: - -```bash -# Show all hooks across a rig -gt hooks [rig] - -# Output: -# 📌 Hooks in gastown: -# -# Polecat/Toast: gt-xyz (feature: Add login) -# Polecat/Alpha: (empty) -# Witness: wisp-abc (patrol cycle) -# Refinery: gt-epic-123 (batch: Q4 features) -# Crew/joe: gt-789 (long: Refactor auth) -# Crew/max: (empty) - -# Show hook for specific agent -gt mol status polecat/Toast -gt mol status witness/ -gt mol status crew/joe -``` - -### Dashboard Data Structure - -```go -type HookStatus struct { - Agent string // Full address (gastown/polecat/Toast) - Role string // polecat, witness, refinery, crew, deacon, mayor - HasWork bool // Is something attached? - AttachedID string // Molecule/issue ID - AttachedTitle string // Human-readable title - AttachedAt time.Time // When attached - IsWisp bool // Ephemeral? - Progress *Progress // Completion percentage -} - -type RigHooks struct { - Rig string - Polecats []HookStatus - Crew []HookStatus - Witness HookStatus - Refinery HookStatus -} - -type TownHooks struct { - Deacon HookStatus - Mayor HookStatus - Rigs []RigHooks -} -``` - -### Proposed: `gt dashboard` - -A unified view of all hooks and work status: - -``` -╔══════════════════════════════════════════════════════════════╗ -║ Gas Town Dashboard ║ -╠══════════════════════════════════════════════════════════════╣ -║ Town: /Users/stevey/gt ║ -║ ║ -║ 🎩 Mayor: (empty) ║ -║ ⛪ Deacon: wisp-patrol (running patrol cycle) ║ -║ ║ -║ 📦 Rig: gastown ║ -║ 👁 Witness: wisp-watch (monitoring 2 polecats) ║ -║ 🏭 Refinery: gt-epic-45 (3/8 issues merged) ║ -║ 🐱 Polecats: ║ -║ Toast: gt-xyz [████████░░] 75% (Implement feature) ║ -║ Alpha: (empty) ║ -║ 👷 Crew: ║ -║ joe: gt-789 [██░░░░░░░░] 20% (Refactor auth) ║ -║ max: (empty) ║ -╚══════════════════════════════════════════════════════════════╝ -``` - ---- - -## Multiple Pins Semantics - -### Question: What if a worker has multiple pinned beads? - -**Current behavior**: Only first pinned bead with matching title is used. - -**Design decision**: **One hook per agent** (enforced) - -Rationale: -- The Propulsion Principle says "if you find something on your hook, run it" -- Multiple hooks would require decision-making about which to run -- Decision-making violates propulsion -- Work queuing belongs in mail or at dispatcher level (Witness/Refinery) - -### Enforcement - -```go -func (b *Beads) GetOrCreateHandoffBead(role string) (*Issue, error) { - existing, err := b.FindHandoffBead(role) - if existing != nil { - return existing, nil // Always return THE ONE handoff bead - } - // Create if not found... -} -``` - -**If somehow multiple exist** (data corruption): -- `gt doctor` should flag as error -- `FindHandoffBead` returns first match (deterministic by ID sort) -- Manual cleanup required - -### Hook Collision on Sling - -When slinging to an occupied hook: - -```bash -$ gt sling feature polecat/Toast -Error: polecat/Toast hook already occupied with gt-xyz -Use --force to replace, or wait for current work to complete. - -$ gt sling feature polecat/Toast --force -⚠️ Detaching gt-xyz from polecat/Toast -📌 Attached gt-abc to polecat/Toast -``` - -**`--force` semantics**: -1. Detach current work (leaves it orphaned in beads) -2. Attach new work -3. Log the replacement for audit - ---- - -## Mail Attachments vs Pinned Attachments - -### Question: What about mails with attached work that aren't pinned? - -**Scenario**: Agent receives mail with `attached_molecule: gt-xyz` in body, -but the mail itself is not pinned, and their hook is empty. - -### Current Protocol - -``` -1. Check hook (gt mol status) - → If work on hook → Run it - -2. Check mail inbox - → If mail has attached work → Mail is the sling delivery mechanism - → The attached work gets pinned to hook automatically - -3. No work anywhere → Idle/await -``` - -### Design Decision: Mail Is Delivery, Hook Is Authority - -**Mail with attachment** = "Here's work for you" -**Hook with attachment** = "This is your current work" - -The sling operation does both: -1. Creates mail notification (optional, for context) -2. Attaches to hook (authoritative) - -**But what if only mail exists?** (manual mail, broken sling): - -| Situation | Expected Behavior | -|-----------|-------------------| -| Hook has work, mail has work | Hook wins. Mail is informational. | -| Hook empty, mail has work | Agent should self-pin from mail. | -| Hook has work, mail empty | Work continues from hook. | -| Both empty | Idle. | - -### Protocol for Manual Work Assignment - -If someone sends mail with attached work but doesn't sling: - -```markdown -## Agent Startup Protocol (Extended) - -1. Check hook: `gt mol status` - - Found work? **Run it.** - -2. Hook empty? Check mail: `gt mail inbox` - - Found mail with `attached_molecule`? - - Self-pin it: `gt mol attach ` - - Then run it. - -3. Nothing? Idle. -``` - -### Self-Pin Command - -```bash -# Agent self-pins work from mail -gt mol attach-from-mail - -# This: -# 1. Reads mail body for attached_molecule field -# 2. Attaches molecule to agent's hook -# 3. Marks mail as read -# 4. Returns control for execution -``` - ---- - -## `gt doctor` Checks - -### Current State - -`gt doctor` doesn't check pinned beads at all. - -### Proposed Checks - -```go -// DoctorCheck definitions for pinned beads -var pinnedBeadChecks = []DoctorCheck{ - { - Name: "hook-singleton", - Description: "Each agent has at most one handoff bead", - Check: checkHookSingleton, - }, - { - Name: "hook-attachment-valid", - Description: "Attached molecules exist and are not closed", - Check: checkHookAttachmentValid, - }, - { - Name: "orphaned-attachments", - Description: "No molecules attached to non-existent hooks", - Check: checkOrphanedAttachments, - }, - { - Name: "stale-attachments", - Description: "Attached molecules not stale (>24h without progress)", - Check: checkStaleAttachments, - }, - { - Name: "hook-agent-mismatch", - Description: "Hook titles match existing agents", - Check: checkHookAgentMismatch, - }, -} -``` - -### Check Details - -#### 1. hook-singleton -``` -✗ Multiple handoff beads for polecat/Toast: - - gt-abc: "Toast Handoff" (created 2025-12-01) - - gt-xyz: "Toast Handoff" (created 2025-12-15) - - Fix: Delete duplicate(s) with `bd close gt-xyz --reason="duplicate hook"` -``` - -#### 2. hook-attachment-valid -``` -✗ Hook attachment points to missing molecule: - Hook: gt-abc (Toast Handoff) - Attached: gt-xyz (not found) - - Fix: Clear attachment with `gt mol detach polecat/Toast` -``` - -#### 3. orphaned-attachments -``` -⚠ Molecule attached but agent doesn't exist: - Molecule: gt-xyz (attached to "Defunct Handoff") - Agent: polecat/Defunct (not found) - - Fix: Re-sling to active agent or close molecule -``` - -#### 4. stale-attachments -``` -⚠ Stale work on hook (48h without progress): - Hook: polecat/Toast - Molecule: gt-xyz (attached 2025-12-21T10:00:00Z) - Last activity: 2025-12-21T14:30:00Z - - Suggestion: Check polecat status, consider nudge or reassignment -``` - -#### 5. hook-agent-mismatch -``` -⚠ Handoff bead for non-existent agent: - Hook: "OldPolecat Handoff" (gt-abc) - Agent: polecat/OldPolecat (no worktree found) - - Fix: Close orphaned hook or recreate agent -``` - -### Implementation - -```go -func (d *Doctor) checkPinnedBeads() []DoctorResult { - results := []DoctorResult{} - - // Get all pinned beads - pinned, _ := d.beads.List(ListOptions{Status: StatusPinned}) - - // Group by title suffix (role) - byRole := groupByRole(pinned) - - // Check singleton - for role, beads := range byRole { - if len(beads) > 1 { - results = append(results, DoctorResult{ - Check: "hook-singleton", - Status: "error", - Message: fmt.Sprintf("Multiple hooks for %s", role), - Details: beads, - }) - } - } - - // Check attachment validity - for _, bead := range pinned { - fields := ParseAttachmentFields(bead) - if fields != nil && fields.AttachedMolecule != "" { - mol, err := d.beads.Show(fields.AttachedMolecule) - if err != nil || mol == nil { - results = append(results, DoctorResult{ - Check: "hook-attachment-valid", - Status: "error", - Message: "Attached molecule not found", - // ... - }) - } - } - } - - // ... other checks - - return results -} -``` - ---- - -## Terminology Decisions - -Based on this analysis, proposed standard terminology: - -| Term | Definition | -|------|------------| -| **Hook** | The pinned bead where work attaches (the attachment point) | -| **Pinned Bead** | A bead with `status: pinned` (never closes) | -| **Handoff Bead** | The specific pinned bead titled "{role} Handoff" | -| **Attachment** | The molecule/issue currently on a hook | -| **Sling** | The act of putting work on an agent's hook | -| **Lead Bead** | The root bead of an attached molecule (synonym: molecule root) | - -**Recommendation**: Use "hook" in user-facing commands and docs, "handoff bead" -in implementation details. - ---- - -## Open Questions - -### 1. Should mail notifications include hook status? - -``` -📬 New work assigned! - -From: witness/ -Subject: Feature work assigned - -Work: gt-xyz (Implement login) -Molecule: feature (4 steps) - -🪝 Hooked to: polecat/Toast - Status: Attached and ready - -Run `gt mol status` to see details. -``` - -**Recommendation**: Yes. Mail should confirm hook attachment succeeded. - -### 2. Should agents be able to self-detach? - -```bash -# Polecat decides work is blocked and gives up -gt mol detach -``` - -**Recommendation**: Yes, but with audit trail. Detachment without completion -should be logged and possibly notify Witness. - -### 3. Multiple rigs with same agent names? - -``` -gastown/polecat/Toast -otherrig/polecat/Toast -``` - -**Current**: Each rig has separate beads, so no collision. -**Future**: If cross-rig visibility needed, full addresses required. - -### 4. Hook persistence during agent recreation? - -When a polecat is killed and recreated: -- Hook bead persists (it's in rig beads, not agent's worktree) -- Old attachment may be stale -- New sling should `--force` or detach first - -**Recommendation**: `gt polecat recreate` should clear hook. - ---- - -## Implementation Roadmap - -### Phase 1: Doctor Checks (immediate) -- [ ] Add `hook-singleton` check -- [ ] Add `hook-attachment-valid` check -- [ ] Add to default doctor run - -### Phase 2: Dashboard Visibility -- [ ] Implement `gt hooks` command -- [ ] Add hook status to `gt status` output -- [ ] Consider `gt dashboard` for full view - -### Phase 3: Protocol Enforcement -- [ ] Add self-pin from mail (`gt mol attach-from-mail`) -- [ ] Audit trail for detach operations -- [ ] Witness notification on abnormal detach - -### Phase 4: Documentation -- [ ] Update role prompt templates with hook protocol -- [ ] Add troubleshooting guide for hook issues -- [ ] Document recovery procedures - ---- - -## Summary - -The pinned bead architecture provides: - -1. **Universal hook per agent** - Every agent has exactly one handoff bead -2. **Title-based discovery** - `{role} Handoff` naming convention -3. **Attachment fields** - `attached_molecule` and `attached_at` in description -4. **Propulsion compliance** - One hook = no decision paralysis -5. **Mail as delivery** - Sling sends notification, attaches to hook -6. **Doctor validation** - Checks for singleton, validity, staleness - -The key insight is that **hooks are authority, mail is notification**. If -they conflict, the hook wins. This maintains the Propulsion Principle: -"If you find something on your hook, YOU RUN IT." diff --git a/docs/polecat-lifecycle.md b/docs/polecat-lifecycle.md deleted file mode 100644 index 39fb6c70..00000000 --- a/docs/polecat-lifecycle.md +++ /dev/null @@ -1,130 +0,0 @@ -# Polecat Lifecycle - -> Polecats restart after each molecule step. This is intentional. - -## Execution Model - -| Phase | What Happens | -|-------|--------------| -| **Spawn** | Worktree created, session started, molecule slung to hook | -| **Step** | Polecat reads hook, executes ONE step, runs `gt mol step done` | -| **Restart** | Session respawns with fresh context, next step on hook | -| **Complete** | Last step done → POLECAT_DONE mail → cleanup wisp created | -| **Cleanup** | Witness verifies git clean, kills session, burns wisp | - -``` -spawn → step → restart → step → restart → ... → complete → cleanup - └──────────────────────────────────────┘ - (fresh session each step) -``` - -## Why Restart Every Step? - -| Reason | Explanation | -|--------|-------------| -| **Atomicity** | Each step completes fully or not at all | -| **No wandering** | Polecat can't half-finish and get distracted | -| **Context fresh** | No accumulation of stale context across steps | -| **Crash recovery** | Restart = re-read hook = continue from last completed step | - -**Trade-off**: Session restart overhead. Worth it for reliability at current cognition levels. - -## Step Packing (Author Responsibility) - -Formula authors must size steps appropriately: - -| Too Small | Too Large | -|-----------|-----------| -| Restart overhead dominates | Context exhaustion mid-step | -| Thrashing | Partial completion, unreliable | - -**Rule of thumb**: A step should use 30-70% of available context. Batch related micro-tasks. - -## The `gt mol step done` Command - -Canonical way to complete a step: - -```bash -gt mol step done -``` - -1. Closes the step in beads -2. Finds next ready step (dependency-aware) -3. Updates hook to next step -4. Respawns pane with fresh session - -**Never use `bd close` directly** - it skips the restart logic. - -## Cleanup: The Finalizer Pattern - -When polecat signals completion: - -``` -POLECAT_DONE mail → Witness creates cleanup wisp → Witness processes wisp → Burn -``` - -The wisp's existence IS the pending cleanup. No explicit queue. - -| Cleanup Step | Verification | -|--------------|--------------| -| Git status | Must be clean | -| Unpushed commits | None allowed | -| Issue state | Closed or deferred | -| Productive work | Commits reference issue (ZFC - Witness judges) | - -Failed cleanup? Leave wisp, retry next cycle. - ---- - -## Evolution Path - -Current design will evolve as model cognition improves: - -| Phase | Refresh Trigger | Who Decides | Witness Load | -|-------|-----------------|-------------|--------------| -| **Now** | Step boundary | Formula (fixed) | High | -| **Spoon-feeding** | Context % + task size | Witness | Medium | -| **Self-managed** | Self-awareness | Polecat | Low | - -### Now (Step-Based Restart) - -- Restart every step, guaranteed -- Conservative, reliable -- `gt mol step done` handles everything - -### Spoon-feeding (Future) - -Requires: Claude Code exposes context usage - -``` -Polecat completes step - → Witness checks: 65% context used - → Next task estimate: 10% context - → Decision: "send another" or "recycle" -``` - -Witness becomes supervisor, not babysitter. - -### Self-Managed (Future) - -Requires: Model cognition threshold + Gas Town patterns in training - -``` -Polecat completes step - → Self-assesses: "I'm at 80%, should recycle" - → Runs gt handoff, respawns -``` - -Polecats become autonomous. Witness becomes auditor. - ---- - -## Key Commands - -| Command | Effect | -|---------|--------| -| `gt mol step done ` | Complete step, restart for next | -| `gt mol status` | Show what's on hook | -| `gt mol progress ` | Show molecule completion state | -| `gt done` | Signal POLECAT_DONE to Witness | -| `gt handoff` | Write notes, respawn (manual refresh) | diff --git a/docs/polecat-wisp-architecture.md b/docs/polecat-wisp-architecture.md deleted file mode 100644 index 0fde9df1..00000000 --- a/docs/polecat-wisp-architecture.md +++ /dev/null @@ -1,333 +0,0 @@ -# Polecat Wisp Architecture: Proto → Wisp → Mol - -> Issue: gt-9g82 - -## Executive Summary - -This document proposes a three-layer architecture for agent work: **Proto → Wisp → Mol**. -The key insight is that agents should run session wisps that wrap their assigned work. -This creates a unified "engineer in a box" pattern that handles onboarding, execution, -and cleanup within a single ephemeral container. - ---- - -## 1. The Three-Layer Model - -``` -┌─────────────────────────────────────────────────────────────────────┐ -│ PROTO LAYER │ -│ ┌───────────────────────────────────────────────────────────────┐ │ -│ │ polecat.md / crew.md (Template/Instructions) │ │ -│ │ - Role identity and context │ │ -│ │ - How to be this type of agent │ │ -│ │ - Workflow patterns │ │ -│ └───────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌───────────────────────────────────────────────────────────────┐ │ -│ │ mol-polecat-session / mol-crew-session │ │ -│ │ - Onboarding step │ │ -│ │ - Execute work step │ │ -│ │ - Cleanup/handoff step │ │ -│ └───────────────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────┘ - │ - │ instantiate - ▼ -┌─────────────────────────────────────────────────────────────────────┐ -│ WISP LAYER │ -│ ┌───────────────────────────────────────────────────────────────┐ │ -│ │ wisp-- │ │ -│ │ Storage: .beads-wisp/ (ephemeral, gitignored) │ │ -│ │ │ │ -│ │ Steps: │ │ -│ │ 1. orient - Load context, read role template │ │ -│ │ 2. handoff - Check for predecessor handoff │ │ -│ │ 3. execute - Run the attached work molecule │ │ -│ │ 4. cleanup - Sync, handoff to successor, exit │ │ -│ └───────────────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────┘ - │ - │ attachment - ▼ -┌─────────────────────────────────────────────────────────────────────┐ -│ MOL LAYER │ -│ ┌───────────────────────────────────────────────────────────────┐ │ -│ │ The Actual Work (Permanent, Auditable) │ │ -│ │ Storage: .beads/ (git-tracked) │ │ -│ │ │ │ -│ │ Could be: │ │ -│ │ - mol-shiny (full workflow) │ │ -│ │ - mol-quick-fix (fast path) │ │ -│ │ - Any custom work molecule │ │ -│ │ - Direct issue (no molecule, just task) │ │ -│ │ - Long-lived epic spanning many sessions │ │ -│ └───────────────────────────────────────────────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────┘ -``` - ---- - -## 2. Agent Types and Their Wisps - -### 2.1 Comparison: Patrol vs Work Agents - -| Aspect | Patrol Wisp (Deacon/Refinery) | Work Wisp (Polecat/Crew) | -|--------|-------------------------------|--------------------------| -| **Lifecycle** | Looping - spawn, execute, burn, repeat | One-shot per session | -| **Inner Work** | Patrol steps (fixed, no attach) | Attached work mol (variable) | -| **Persistence** | Burns between cycles | Burns on session end | -| **Session** | Persists across wisps | Polecat: terminates; Crew: cycles | -| **Audit Trail** | Digests only (summaries) | Work mol is permanent | - -### 2.2 Polecat Wisp (Ephemeral Worker) - -Polecats are transient workers that: -- Get spawned for specific work -- Self-destruct on completion -- Have no persistent identity - -``` -Polecat Session Flow: -───────────────────── -spawn → orient → handoff(read) → execute → cleanup → TERMINATE - │ - └──▶ [attached work mol] -``` - -### 2.3 Crew Worker Wisp (Persistent Worker) - -Crew workers are long-lived agents that: -- Maintain persistent identity across sessions -- Work on long-lived molecules (epics spanning days/weeks) -- Hand off to successor sessions (themselves) -- **Can work autonomously overnight** if attached mol exists - -``` -Crew Worker Session Flow: -───────────────────────── -start → orient → handoff(read) → check attachment - │ - ┌──────────────┴──────────────┐ - │ │ - ATTACHED NO ATTACHMENT - │ │ - ▼ ▼ - ┌───────────────┐ ┌───────────────┐ - │ AUTO-CONTINUE │ │ AWAIT INPUT │ - │ (no prompt) │ │ (human directs)│ - └───────┬───────┘ └───────────────┘ - │ - ▼ - execute mol - │ - ▼ - cleanup → handoff(write) → END SESSION - │ - ▼ - (successor picks up) -``` - -**Key insight**: If there's an attached mol, crew workers continue working -without awaiting input. This enables autonomous overnight work on long molecules. - ---- - -## 3. The Onboard-Execute-Cleanup Pattern - -### 3.1 mol-polecat-session - -```markdown -## Step: orient -Read polecat.md protocol. Understand: -- Your identity (rig/polecat-name) -- The beads system -- Exit strategies -- Handoff protocols - -Load context: -- gt prime -- gt mail inbox - -## Step: handoff-read -Check for predecessor handoff mail. -If found, load context from previous session. - -## Step: execute -Find attached work: -- gt mol status (shows what's on hook) -- bd show - -Run the attached work molecule to completion. -Continue until reaching exit-decision step. - -## Step: cleanup -1. Sync state: bd sync, git push -2. Close or defer work mol based on exit type -3. Burn this session wisp -4. Request shutdown from Witness -``` - -### 3.2 mol-crew-session (Light Harness) - -```markdown -## Step: orient -Read crew.md protocol. Load context: -- gt prime -- Identify self (crew member name, rig) - -## Step: handoff-read -Check inbox for 🤝 HANDOFF messages: -- gt mail inbox -- If found: read and load predecessor context - -## Step: check-attachment -Look for pinned work: -- bd list --pinned --assignee= --status=in_progress -- gt mol status - -If attachment found: - → Proceed to execute (no human input needed) -If no attachment: - → Await user instruction - -## Step: execute -Work the attached molecule: -- bd ready --parent= -- Continue from last completed step -- Work until context full or natural stopping point - -## Step: cleanup -Before session ends: -1. git status / git push -2. bd sync -3. Write handoff mail to self: - gt mail send -s "🤝 HANDOFF: " -m "

" -4. Session ends (successor will pick up) -``` - ---- - -## 4. Autonomous Overnight Work - -### 4.1 The Vision - -With session wisps, crew workers can work autonomously: - -``` -Night 1, Session 1: - Human: "Work on gt-abc (big feature epic)" - Crew: Attaches gt-abc, works 2 hours, context full - Crew: Writes handoff, ends session - -Night 1, Session 2 (auto-spawned): - Crew: Reads handoff, finds attached gt-abc - Crew: AUTO-CONTINUES (no human needed) - Crew: Works 2 more hours, context full - Crew: Writes handoff, ends session - -... repeat through the night ... - -Morning: - Human: Checks progress on gt-abc - Crew: "Completed 15 of 23 subtasks overnight" -``` - -### 4.2 Requirements for Autonomous Work - -1. **Attached mol** - Work must be pinned/attached -2. **Clear exit conditions** - Mol defines when to stop -3. **Session cycling** - Auto-spawn successor sessions -4. **Handoff protocol** - Each session writes handoff for next - -### 4.3 Safety Rails - -- Work mol defines scope (can't go off-rails) -- Each session is bounded (context limit) -- Handoffs create audit trail -- Human can intervene at any session boundary - ---- - -## 5. Data Structures - -### 5.1 Session Wisp - -```json -{ - "id": "wisp-max-2024-12-22T15:00:00Z", - "type": "wisp", - "title": "Crew Session: max", - "status": "in_progress", - "current_step": "execute", - "agent_type": "crew", - "agent_name": "max", - "attached_mol": "gt-abc123", - "attached_at": "2024-12-22T15:01:00Z" -} -``` - -### 5.2 Attachment Mechanism - -```bash -# Attach a work mol to current session wisp -gt mol attach - -# Check current attachment -gt mol status - -# Detach (human override) -gt mol detach -``` - ---- - -## 6. Implementation Phases - -### Phase 1: Core Infrastructure (P0) - -1. Create `mol-crew-session.formula.json` in `.beads/formulas/` -2. Create `mol-polecat-session.formula.json` in `.beads/formulas/` -3. Add wisp attachment mechanism to beads -4. Update spawn.go for polecat session wisps - -### Phase 2: Crew Worker Integration (P0) - -1. Update crew startup hook to instantiate session wisp -2. Implement auto-continue logic (if attached → work) -3. Update crew.md template for session wisp model - -### Phase 3: Session Cycling (P1) - -1. Add session successor spawning -2. Implement context-full detection -3. Auto-handoff on session end - -### Phase 4: Monitoring & Safety (P1) - -1. Witness monitors session wisps -2. Add emergency stop mechanism -3. Progress reporting between sessions - ---- - -## 7. Open Questions - -1. **Session cycling trigger**: Context percentage? Time limit? Both? -2. **Emergency stop**: How does human halt autonomous work? -3. **Progress visibility**: Dashboard for overnight work progress? -4. **Mol attachment UI**: How does human attach/detach work? - ---- - -## 8. Summary - -The session wisp architecture enables: - -- **Separation of concerns**: How to work (proto) vs What to work on (mol) -- **Session continuity**: Handoffs preserve context across sessions -- **Autonomous work**: Crew workers can work overnight on attached mols -- **Unified pattern**: Same model for polecats and crew workers -- **Audit trail**: Work mol is permanent, session wisps are ephemeral - -The key unlock is: **if there's attached work, continue without prompting**. -This transforms crew workers from interactive assistants to autonomous workers. diff --git a/docs/prompts.md b/docs/prompts.md deleted file mode 100644 index 21e0fa3c..00000000 --- a/docs/prompts.md +++ /dev/null @@ -1,441 +0,0 @@ -# Gas Town Prompt Architecture - -This document defines the canonical prompt system for Go Gas Town (GGT). Prompts are the personality and competence instructions that shape each agent's behavior. - -## Design Principles - -1. **Role-specific context**: Each agent type gets tailored prompts for their responsibilities -2. **Recovery-friendly**: Prompts include recovery instructions for context loss -3. **Command-focused**: Essential commands prominently featured -4. **Checklist-driven**: Critical workflows enforced via checklists -5. **Handoff-aware**: Session cycling built into prompt design - -## Prompt Categories - -### 1. Role Prompts (Primary Context) - -Delivered via `gt prime` command. These establish agent identity and capabilities. - -| Role | Template | Purpose | -|------|----------|---------| -| Mayor | `mayor.md` | Global coordination, work dispatch, cross-rig decisions | -| Witness | `witness.md` | Worker monitoring, nudging, pre-kill verification, session cycling | -| Refinery | `refinery.md` | Merge queue processing, PR review, integration | -| Polecat | `polecat.md` | Implementation work on assigned issues | -| Crew | `crew.md` | Overseer's personal workspace, user-managed, persistent identity | -| Unknown | `unknown.md` | Fallback when role detection fails | - -### 2. Mail Templates (Structured Messages) - -Parseable messages for worker coordination. - -| Template | Flow | Purpose | -|----------|------|---------| -| `BATCH_STARTED` | Refinery → Mayor | Batch initialized, workers ready | -| `WORKER_READY` | Worker → Refinery | Worker setup complete | -| `WORK_COMPLETE` | Worker → Refinery | Task finished, ready for merge | -| `MERGE_CONFLICT` | Refinery → Worker/Mayor | Merge failed, needs resolution | -| `BATCH_COMPLETE` | Refinery → Mayor | All tasks done | -| `BATCH_FAILED` | Refinery → Mayor | Batch failed, needs intervention | - -### 3. Spawn Injection (Work Assignment) - -Prompts injected when assigning work to agents. - -| Context | Content | -|---------|---------| -| New polecat | Issue details, commands, completion checklist | -| Reused polecat | Compact issue summary, `bd show` reminder | -| Ephemeral worker | Issue details with read-only beads instructions | - -### 4. Lifecycle Templates (Session Management) - -Templates for session cycling and handoffs. - -| Template | Purpose | -|----------|---------| -| `HANDOFF` | Session-to-self handoff with state capture | -| `ESCALATION` | Worker → Witness → Mayor escalation | -| `NUDGE` | Witness → Worker progress reminder | - -## PGT Prompt Inventory - -### Role Prompts (templates/*.md.j2) - -**mayor.md.j2** (~100 lines) -- Role identification and scope -- Workspace locations (`~/ai/mayor/rigs//`) -- Directory structure diagram -- Essential commands (status, inbox, spawn, refinery) -- Working on code and beads section -- Delegation rules -- Session end checklist with handoff protocol - -**polecat.md.j2** (~120 lines) -- Role identification with rig/polecat name injection -- Ephemeral mode conditional sections -- Critical "work in YOUR directory only" warning -- Essential commands (finding work, working, completing) -- Ephemeral-specific status reporting commands -- Required completion checklist (bd close FIRST) -- "If you get stuck" section - -**refinery.md.j2** (~70 lines) -- Role identification with rig name -- Directory structure showing refinery/rig/ location -- Polecat management commands -- Work management and sync commands -- Workflow steps (inbox → ready → spawn → monitor → report) -- Worker management commands -- Session end checklist - -**unknown.md.j2** (~20 lines) -- Location unknown message -- Navigation suggestions -- Orientation commands - -### Static Fallbacks (prime_cmd.py) - -Each role has a `_get__context_static()` function providing fallback prompts when Jinja2 is unavailable. These are simplified versions of the templates. - -### Mail Templates (mail/templates.py) - -~450 lines of structured message generators: -- Metadata format: `[KEY]: value` lines for parsing -- Human-readable context below metadata -- Priority levels (normal, high) -- Message types (NOTIFICATION, TASK) - -### Spawn Injection (spawn.py) - -**send_initial_prompt()** (~40 lines) -- Issue ID and title -- Description (if available) -- Additional instructions -- Command reference (bd show, bd update, bd close) - -**Direct injection prompts** (inline) -- Compact single-line format for tmux injection -- Issue ID, title, extra prompt, completion reminder - -## Gap Analysis - -### Missing Prompts - -| Gap | Priority | Notes | -|-----|----------|-------| -| **Witness role prompt** | P0 | New role, no prompts exist | -| Session handoff template | P1 | Currently ad-hoc in agent logic | -| Escalation message template | P1 | No standard format | -| Nudge message template | P2 | Witness → Worker reminders | -| Direct landing workflow | P2 | Mayor bypass of Refinery | - -### Missing Features - -| Feature | Priority | Notes | -|---------|----------|-------| -| Prompt versioning | P2 | No way to track prompt changes | -| Prompt testing | P3 | No validation that prompts work | -| Dynamic context | P2 | Templates don't adapt to agent state | - -## GGT Architecture Recommendations - -### 1. Prompt Storage - -Templates are embedded in the Go binary via `//go:embed`: - -``` -internal/templates/ -├── roles/ -│ ├── mayor.md.tmpl -│ ├── witness.md.tmpl -│ ├── refinery.md.tmpl -│ ├── polecat.md.tmpl -│ ├── crew.md.tmpl -│ └── deacon.md.tmpl -└── messages/ - ├── spawn.md.tmpl - ├── nudge.md.tmpl - ├── escalation.md.tmpl - └── handoff.md.tmpl -``` - -### 2. Template Engine - -Use Go's `text/template` with a simple context struct: - -```go -type PromptContext struct { - Role string - RigName string - PolecatName string - Transient bool - IssueID string - IssueTitle string - // ... additional fields -} -``` - -### 3. Prompt Registry - -```go -type PromptRegistry struct { - roles map[string]*template.Template - mail map[string]*template.Template - spawn map[string]*template.Template - lifecycle map[string]*template.Template -} - -func (r *PromptRegistry) Render(category, name string, ctx PromptContext) (string, error) -``` - -### 4. CLI Integration - -```bash -gt prime # Auto-detect role, render appropriate prompt -gt prime --mayor # Force mayor prompt -gt prime --witness # Force witness prompt -gt prompt render / --context '{"rig": "foo"}' -gt prompt list # List all available prompts -gt prompt validate # Check all prompts parse correctly -``` - -## Witness Prompt Design - -The Witness is a new role - the per-rig "pit boss" who monitors workers. Here's the canonical prompt: - -```markdown -# Gastown Witness Context - -> **Recovery**: Run `gt prime` after compaction, clear, or new session - -## Your Role: WITNESS (Pit Boss for {{ rig_name }}) - -You are the per-rig worker monitor. You watch polecats, nudge them toward completion, -verify clean git state before kills, and escalate stuck workers to the Mayor. - -**You do NOT do implementation work.** Your job is oversight, not coding. - -## Your Workspace - -You work from: `~/ai/{{ rig_name }}/witness/` - -You monitor polecats in: `~/ai/{{ rig_name }}/polecats/*/` - -## Core Responsibilities - -1. **Monitor workers**: Track polecat health and progress -2. **Nudge**: Prompt slow workers toward completion -3. **Pre-kill verification**: Ensure git state is clean before killing sessions -4. **Session lifecycle**: Kill sessions, update worker state -5. **Self-cycling**: Hand off to fresh session when context fills -6. **Escalation**: Report stuck workers to Mayor - -**Key principle**: You own ALL per-worker cleanup. Mayor is never involved in routine worker management. - -## Essential Commands - -### Monitoring -- `gt polecats {{ rig_name }}` - List all polecats and status -- `gt polecat status ` - Detailed polecat status -- `gt polecat git-state ` - Check git cleanliness - -### Worker Management -- `gt polecat nudge "message"` - Send nudge to worker -- `gt polecat kill ` - Kill worker session (after verification!) -- `gt polecat wake ` - Mark worker as active -- `gt polecat sleep ` - Mark worker as inactive - -### Communication -- `gt inbox` - Check your messages -- `gt send mayor/ -s "Subject" -m "Message"` - Escalate to Mayor -- `gt send {{ rig_name }}/ -s "Subject" -m "Message"` - Message worker - -### Beads -- `bd list --status=in_progress` - Active work in this rig -- `bd show ` - Issue details - -## Pre-Kill Verification Checklist - -Before killing ANY polecat session, verify: - -``` -[ ] 1. gt polecat git-state # Must be clean -[ ] 2. Check for uncommitted work # git status in polecat dir -[ ] 3. Check for unpushed commits # git log origin/main..HEAD -[ ] 4. Verify issue closed # bd show shows closed -``` - -If git state is dirty: -1. Nudge the worker to clean up -2. Wait for response (give 2-3 nudges max) -3. If still dirty after 3 nudges → Escalate to Mayor - -## Nudge Protocol - -When a worker seems stuck or slow: - -1. **First nudge** (gentle): "How's progress on ? Need any help?" -2. **Second nudge** (direct): "Please wrap up soon. What's blocking you?" -3. **Third nudge** (final): "Final check on . If blocked, I'll escalate to Mayor." -4. **Escalate**: If no progress after 3 nudges, send escalation to Mayor - -Use: `gt polecat nudge ""` - -## Escalation Template - -When escalating to Mayor: - -``` -gt send mayor/ -s "Escalation: stuck on " -m " -Worker: -Issue: -Status: - -Attempts: -- Nudge 1: - -- Nudge 2: - -- Nudge 3: - - -Git state: - -Recommendation: -" -``` - -## Session Self-Cycling - -When your context fills up: - -1. Capture current state (active workers, pending nudges, recent events) -2. Send handoff to yourself: - ``` - gt send {{ rig_name }}/witness -s "🤝 HANDOFF: Witness session cycle" -m " - Active workers: - Pending nudges: - Recent escalations: - Notes: - " - ``` -3. Exit cleanly - -## Session End Checklist - -``` -[ ] gt polecats {{ rig_name }} (check all worker states) -[ ] Review any pending nudges -[ ] Escalate any truly stuck workers -[ ] HANDOFF if work incomplete: - gt send {{ rig_name }}/witness -s "🤝 HANDOFF: ..." -m "..." -``` -``` - -## Crew Prompt Design - -Crew workers are the overseer's personal workspaces - a new role that differs from polecats: - -```markdown -# Gas Town Crew Worker Context - -> **Recovery**: Run `gt prime` after compaction, clear, or new session - -## Your Role: CREW WORKER ({{ name }} in {{ rig }}) - -You are a **crew worker** - the overseer's (human's) personal workspace within the {{ rig }} rig. -Unlike polecats which are witness-managed and transient, you are: - -- **Persistent**: Your workspace is never auto-garbage-collected -- **User-managed**: The overseer controls your lifecycle, not the Witness -- **Long-lived identity**: You keep your name ({{ name }}) across sessions -- **Integrated**: Mail and handoff mechanics work just like other Gas Town agents - -**Key difference from polecats**: No one is watching you. You work directly with the overseer. - -## Your Workspace - -You work from: `{{ workspace_path }}` - -This is a full git clone of the project repository. - -## Essential Commands - -### Finding Work -- `gt mail inbox` - Check your messages -- `bd ready` - Available issues (if beads configured) -- `bd list --status=in_progress` - Your active work - -### Working -- `bd update --status=in_progress` - Claim an issue -- `bd show ` - View issue details -- Standard git workflow (status, add, commit, push) - -### Completing Work -- `bd close ` - Close the issue -- `bd sync` - Sync beads changes - -## Context Cycling (Handoff) - -When context fills up, send a handoff mail to yourself: - -```bash -gt mail send {{ rig }}/{{ name }} -s "HANDOFF: Work in progress" -m " -Working on: -Branch: -Status: -Next steps: -" -``` - -Or use: `gt crew refresh {{ name }}` - -## No Witness Monitoring - -Unlike polecats, crew workers have no Witness oversight: -- No automatic nudging -- No pre-kill verification -- No escalation on blocks -- No automatic cleanup - -**You are responsible for**: Managing progress, asking for help, keeping git clean. - -## Session End Checklist - -``` -[ ] git status / git push -[ ] bd sync (if configured) -[ ] Check inbox -[ ] HANDOFF if incomplete -``` -``` - -## Implementation Plan - -### Phase 1: Core Role Prompts -1. Port mayor.md.j2 to Go template -2. Create witness.md (new!) -3. Port refinery.md.j2 -4. Port polecat.md.j2 -5. Create crew.md (new!) -6. Port unknown.md.j2 - -### Phase 2: Mail Templates -1. Define mail template format in Go -2. Port worker coordination templates -3. Add handoff and escalation templates - -### Phase 3: Spawn & Lifecycle -1. Port spawn injection prompts -2. Add lifecycle templates (nudge, escalation) -3. Integrate with `gt sling` command - -### Phase 4: CLI & Validation -1. Implement `gt prime` with role detection -2. Add `gt prompt` subcommands -3. Add prompt validation/testing - -## Related Issues - -- `gt-u1j`: Port Gas Town to Go (parent epic) -- `gt-f9x`: Town & Rig Management -- `gt-cik`: Overseer Crew: User-managed persistent workspaces -- `gt-iib`: Decentralized rig structure (affects prompt paths) diff --git a/docs/propulsion-principle.md b/docs/propulsion-principle.md deleted file mode 100644 index 912f1630..00000000 --- a/docs/propulsion-principle.md +++ /dev/null @@ -1,282 +0,0 @@ -# The Propulsion Principle - -> **Status**: Design document (experimental) -> **See also**: [sling-design.md](sling-design.md) for implementation details -> **See also**: [molecular-chemistry.md](molecular-chemistry.md) for the full Rig/Cook/Run lifecycle - -## The Core Idea - -We're trying a simple rule for agent behavior: - -> **If you find something on your hook, YOU RUN IT.** - -No decisions. No "should I?" No discretionary pondering. Just ignition. - -``` -Hook has work → Work happens. -``` - -That's the whole engine. Everything else is plumbing. - -## Why It Works - -The Propulsion Principle works because of three interlocking design decisions: - -### 1. Agents Are Stateless - -Agents have no memory between sessions. When a session starts, the agent has -no inherent knowledge of what it was doing before, what the current project -state is, or what decisions led to its existence. - -This sounds like a limitation, but it's actually the foundation of resilience. -Stateless agents can: -- Be restarted at any time without data loss -- Survive context compaction (the agent re-reads its state) -- Hand off to new sessions seamlessly -- Recover from crashes without corruption - -### 2. Work Is Molecule-Driven - -All work in Gas Town follows the **Rig → Cook → Run** lifecycle: -- **Rig**: Compose workflow formulas (YAML source files) -- **Cook**: Transform formulas into executable protos (expand macros, apply aspects) -- **Run**: Agents execute the cooked workflow - -A molecule (proto, mol, or wisp) defines: -- What steps need to happen -- What order they happen in (via dependencies) -- What each step should accomplish - -The agent doesn't decide what to do. The molecule tells it. The agent's job is -execution, not planning. See [molecular-chemistry.md](molecular-chemistry.md) -for the full lifecycle. - -### 3. Hooks Deliver Work - -Work arrives on an agent's **hook** - a pinned molecule or assigned issue that -represents "your current work." When an agent wakes up: - -1. Check the hook -2. Found something? **Execute it.** -3. Nothing? Check mail for new assignments. -4. Repeat. - -The hook eliminates decision-making about what to work on. If it's on your hook, -it's your work. Run it. - -## The Sling Lifecycle - -The **sling** operation puts work on an agent's hook. This is the **Run** phase -of the Rig → Cook → Run lifecycle (formulas have already been cooked into protos): - -``` -┌─────────────────────────────────────────────────────────┐ -│ gt sling lifecycle │ -├─────────────────────────────────────────────────────────┤ -│ │ -│ 1. POUR (if proto) 2. ASSIGN 3. PIN │ -│ proto → mol/wisp mol → agent → hook │ -│ │ -│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ -│ │ Proto │ ────────► │Mol/Wisp │ ─────► │ Hook │ │ -│ │(cooked) │ pour │(instance)│ assign │(pinned) │ │ -│ └─────────┘ └─────────┘ └─────────┘ │ -│ │ │ -│ agent wakes │ -│ │ │ -│ ▼ │ -│ ┌─────────┐ │ -│ │ IGNITION│ │ -│ └─────────┘ │ -└─────────────────────────────────────────────────────────┘ -``` - -**Pour** instantiates a proto into a running mol (persistent) or wisp (ephemeral). -This is a phase transition from the Cook output to the Run phase. - -**Key insight**: The agent never decides *whether* to run. The molecule tells -it *what* to do. It executes until complete, then checks the hook again. - -## Agent Startup Protocol - -Every agent follows the same startup protocol: - -```bash -# 1. Check your hook -gt mol status # What's on my hook? - -# 2. Found something? -# Output tells you exactly what to do. -# Follow the molecule phases. - -# 3. Nothing on hook? -gt mail inbox # Check for new assignments - -# 4. Repeat -``` - -**Old way** (too much thinking): -```bash -gt mail inbox -if has_molecule; then - gt molecule instantiate ... - # figure out what to do... -fi -``` - -**New way** (propulsion): -```bash -gt mol status # What's on my hook? -# Just follow the molecule phases -``` - -The difference is profound: the old way requires the agent to understand its -situation and make decisions. The new way requires only execution. - -## The Steam Engine Metaphor - -Gas Town uses steam engine vocabulary throughout. The full lifecycle is -**Rig → Cook → Run**: - -| Metaphor | Gas Town | Lifecycle Phase | Description | -|----------|----------|-----------------|-------------| -| **Recipe** | Formulas | Rig (source) | YAML files that compose workflows | -| **Fuel** | Proto molecules | Cook (artifact) | Cooked templates ready to instantiate | -| **Steam** | Wisps/Mols | Run (execution) | Active execution traces | -| **Distillate** | Digests | (post-Run) | Condensed permanent records | -| **Burn** | `bd mol burn` | (post-Run) | Discard without record | -| **Squash** | `bd mol squash` | (post-Run) | Compress into digest | - -Claude is fire. Claude Code is a Steam engine. Gas Town is a Steam Train, with -Beads as the tracks. Wisps are steam vapors that dissipate after work is done. - -The Propulsion Principle is the physics that makes the engine go: -**Hook has work → Work happens.** - -## Examples - -### Good: Following the Principle - -```markdown -## Polecat Startup - -1. Run `gt prime` to load context -2. Check `gt mol status` for pinned work -3. Found molecule? Execute each step in order. -4. Complete? Run `gt done` to submit to merge queue. -5. Request shutdown. You're ephemeral. -``` - -### Good: Witness Patrol - -```markdown -## Witness Cycle - -1. Bond a wisp molecule for this patrol cycle -2. Execute patrol steps (check polecats, check refinery) -3. Squash the wisp when done (creates digest) -4. Sleep until next cycle -5. Repeat forever -``` - -### Anti-Pattern: Decision Paralysis - -```markdown -## DON'T DO THIS - -1. Wake up -2. Think about what I should do... -3. Look at various issues and prioritize them -4. Decide which one seems most important -5. Start working on it -``` - -This violates propulsion. If there's nothing on your hook, you check mail or -wait. You don't go looking for work to decide to do. Work is *slung* at you. - -### Anti-Pattern: Ignoring the Hook - -```markdown -## DON'T DO THIS - -1. Wake up -2. See molecule on my hook -3. But I notice a more interesting issue over there... -4. Work on that instead -``` - -If it's on your hook, you run it. Period. The hook is not a suggestion. - -### Anti-Pattern: Partial Execution - -```markdown -## DON'T DO THIS - -1. Wake up -2. See molecule with 5 steps -3. Complete step 1 -4. Get bored, request shutdown -5. Leave steps 2-5 incomplete -``` - -Molecules are executed to completion. If you can't finish, you squash or burn -explicitly. You don't just abandon mid-flight. - -## Why Not Just Use TODO Lists? - -LLM agents have short memories. A TODO list in the prompt will be forgotten -during context compaction. A molecule in beads survives indefinitely because -it's stored outside the agent's context. - -**Molecules are external TODO lists that persist across sessions.** - -This is the secret to autonomous operation: the agent's instructions survive -the agent's death. New agent, same molecule, same progress point. - -## Relationship to Nondeterministic Idempotence - -The Propulsion Principle enables **nondeterministic idempotence** - the property -that any workflow will eventually complete correctly, regardless of which agent -runs which step, and regardless of crashes or restarts. - -| Property | How Propulsion Enables It | -|----------|---------------------------| -| **Deterministic structure** | Molecules define exact steps | -| **Nondeterministic execution** | Any agent can run any ready step | -| **Idempotent progress** | Completed steps stay completed | -| **Crash recovery** | Agent dies, molecule persists | -| **Session survival** | Restart = re-read hook = continue | - -## Implementation Status - -The Propulsion Principle is implemented via three commands: - -| Command | Action | Context | Use Case | -|---------|--------|---------|----------| -| `gt hook ` | Attach only | Preserved | Assign work for later | -| `gt sling ` | Attach + run | Preserved | Kick off work immediately | -| `gt handoff ` | Attach + restart | Fresh | Restart with new context | - -The algebra: -``` -gt sling = gt hook + gt nudge "start working" -gt handoff = gt hook + restart (GUPP kicks in) -``` - -See [sling-design.md](sling-design.md) for detailed command reference. - -## Summary - -1. **One rule**: If you find something on your hook, you run it. -2. **Stateless agents**: No memory between sessions - molecules provide continuity. -3. **Molecule-driven**: Work is defined by molecules, not agent decisions. -4. **Hook delivery**: Work arrives via sling, sits on hook until complete. -5. **Just execute**: No thinking about whether. Only doing. - -The Propulsion Principle is what makes Gas Town work as an autonomous, -distributed, crash-resilient execution engine. It's the physics of the steam -train. - -``` -Hook has work → Work happens. -``` diff --git a/docs/reference.md b/docs/reference.md new file mode 100644 index 00000000..ef66fb56 --- /dev/null +++ b/docs/reference.md @@ -0,0 +1,280 @@ +# Gas Town Reference + +Technical reference for Gas Town internals. Read the README first. + +## Directory Structure + +``` +~/gt/ Town root +├── .beads/ Town-level beads (hq-* prefix) +├── mayor/ Mayor config +│ └── town.json +└── / Project container (NOT a git clone) + ├── config.json Rig identity + ├── .beads/ → mayor/rig/.beads + ├── .repo.git/ Bare repo (shared by worktrees) + ├── mayor/rig/ Mayor's clone (canonical beads) + ├── refinery/rig/ Worktree on main + ├── witness/ No clone (monitors only) + ├── crew// Human workspaces + └── polecats// Worker worktrees +``` + +**Key points:** +- Rig root is a container, not a clone +- `.repo.git/` is bare - refinery and polecats are worktrees +- Mayor clone holds canonical `.beads/`, others inherit via redirect + +## Beads Routing + +Gas Town routes beads commands based on issue ID prefix. You don't need to think +about which database to use - just use the issue ID. + +```bash +bd show gt-xyz # Routes to gastown rig's beads +bd show hq-abc # Routes to town-level beads +bd show wyv-123 # Routes to wyvern rig's beads +``` + +**How it works**: Routes are defined in `~/gt/.beads/routes.jsonl`. Each rig's +prefix maps to its beads location (the mayor's clone in that rig). + +| Prefix | Routes To | Purpose | +|--------|-----------|---------| +| `hq-*` | `~/gt/.beads/` | Mayor mail, cross-rig coordination | +| `gt-*` | `~/gt/gastown/mayor/rig/.beads/` | Gastown project issues | +| `wyv-*` | `~/gt/wyvern/mayor/rig/.beads/` | Wyvern project issues | + +Debug routing: `BD_DEBUG_ROUTING=1 bd show ` + +## Configuration + +### Rig Config (`config.json`) +```json +{ + "type": "rig", + "name": "myproject", + "git_url": "https://github.com/...", + "beads": { "prefix": "mp" } +} +``` + +### Settings (`settings/config.json`) +```json +{ + "theme": "desert", + "max_workers": 5, + "merge_queue": { "enabled": true } +} +``` + +### Runtime (`.runtime/` - gitignored) +Process state, PIDs, ephemeral data. + +## Formula Format + +```toml +formula = "name" +type = "workflow" # workflow | expansion | aspect +version = 1 +description = "..." + +[vars.feature] +description = "..." +required = true + +[[steps]] +id = "step-id" +title = "{{feature}}" +description = "..." +needs = ["other-step"] # Dependencies +``` + +**Composition:** +```toml +extends = ["base-formula"] + +[compose] +aspects = ["cross-cutting"] + +[[compose.expand]] +target = "step-id" +with = "macro-formula" +``` + +## Molecule Lifecycle + +``` +Formula (source TOML) ─── "Ice-9" + │ + ▼ bd cook +Protomolecule (frozen template) ─── Solid + │ + ├─▶ bd pour ──▶ Mol (persistent) ─── Liquid ──▶ bd squash ──▶ Digest + │ + └─▶ bd wisp ──▶ Wisp (ephemeral) ─── Vapor ──┬▶ bd squash ──▶ Digest + └▶ bd burn ──▶ (gone) +``` + +**Note**: Wisps are stored in `.beads/` with an ephemeral flag - they're not +persisted to JSONL. They exist only in memory during execution. + +## Molecule Commands + +```bash +# Formulas +bd formula list # Available formulas +bd formula show # Formula details +bd cook # Formula → Proto + +# Molecules +bd mol list # Available protos +bd mol show # Proto details +bd pour # Create mol +bd wisp # Create wisp +bd mol bond # Attach to existing mol +bd mol squash # Condense to digest +bd mol burn # Discard wisp +``` + +## Agent Lifecycle + +### Polecat Shutdown +``` +1. Complete work steps +2. bd mol squash (create digest) +3. Submit to merge queue +4. gt handoff (request shutdown) +5. Wait for Witness to kill session +6. Witness removes worktree + branch +``` + +### Session Cycling +``` +1. Agent notices context filling +2. gt handoff (sends mail to self) +3. Manager kills session +4. Manager starts new session +5. New session reads handoff mail +``` + +## Environment Variables + +| Variable | Purpose | +|----------|---------| +| `BEADS_DIR` | Point to shared beads database | +| `BEADS_NO_DAEMON` | Required for worktree polecats | +| `GT_TOWN_ROOT` | Override town root detection | + +## CLI Reference + +### Town Management +```bash +gt install [path] # Create town +gt install --git # With git init +gt doctor # Health check +gt doctor --fix # Auto-repair +``` + +### Rig Management +```bash +gt rig add --remote= +gt rig list +gt rig remove +``` + +### Work Assignment +```bash +gt sling # Assign to polecat +gt sling --molecule= +``` + +### Communication +```bash +gt mail inbox +gt mail read +gt mail send -s "Subject" -m "Body" +gt mail send --human -s "..." # To overseer +``` + +### Sessions +```bash +gt handoff # Request cycle (context-aware) +gt handoff --shutdown # Terminate (polecats) +gt session stop / +gt peek # Check health +gt nudge # Wake stuck worker +``` + +### Emergency +```bash +gt stop --all # Kill all sessions +gt stop --rig # Kill rig sessions +``` + +## Beads Commands (bd) + +```bash +bd ready # Work with no blockers +bd list --status=open +bd list --status=in_progress +bd show +bd create --title="..." --type=task +bd update --status=in_progress +bd close +bd dep add # child depends on parent +bd sync # Push/pull changes +``` + +## Patrol Agents + +Deacon, Witness, and Refinery run continuous patrol loops using wisps: + +| Agent | Patrol Molecule | Responsibility | +|-------|-----------------|----------------| +| **Deacon** | `mol-deacon-patrol` | Agent lifecycle, plugin execution, health checks | +| **Witness** | `mol-witness-patrol` | Monitor polecats, nudge stuck workers | +| **Refinery** | `mol-refinery-patrol` | Process merge queue, review PRs | + +``` +1. bd wisp mol--patrol +2. Execute steps (check workers, process queue, run plugins) +3. bd mol squash (or burn if routine) +4. Loop +``` + +## Plugin Molecules + +Plugins are molecules with specific labels: + +```json +{ + "id": "mol-security-scan", + "labels": ["template", "plugin", "witness", "tier:haiku"] +} +``` + +Patrol molecules bond plugins dynamically: +```bash +bd mol bond mol-security-scan $PATROL_ID --var scope="$SCOPE" +``` + +## Common Issues + +| Problem | Solution | +|---------|----------| +| Agent in wrong directory | Check cwd, `gt doctor` | +| Beads prefix mismatch | Check `bd show` vs rig config | +| Worktree conflicts | Ensure `BEADS_NO_DAEMON=1` for polecats | +| Stuck worker | `gt nudge`, then `gt peek` | +| Dirty git state | Commit or discard, then `gt handoff` | + +## Architecture Notes + +**Bare repo pattern**: `.repo.git/` is bare (no working dir). Refinery and polecats are worktrees sharing refs. Polecat branches visible to refinery immediately. + +**Beads as control plane**: No separate orchestrator. Molecule steps ARE beads issues. State transitions are git commits. + +**Nondeterministic idempotence**: Any worker can continue any molecule. Steps are atomic checkpoints in beads. + + diff --git a/docs/rig-cook-run.md b/docs/rig-cook-run.md deleted file mode 100644 index 641f96d1..00000000 --- a/docs/rig-cook-run.md +++ /dev/null @@ -1,14 +0,0 @@ -# Rig, Cook, Run - -> **This document has been consolidated into [molecular-chemistry.md](molecular-chemistry.md).** -> -> See the following sections: -> - [The Work Lifecycle: Rig → Cook → Run](molecular-chemistry.md#the-work-lifecycle-rig--cook--run) -> - [The Complete Artifact Graph](molecular-chemistry.md#the-complete-artifact-graph) -> - [Two Composition Operators](molecular-chemistry.md#two-composition-operators) -> - [Formulas: The Source Layer](molecular-chemistry.md#formulas-the-source-layer) -> - [The Polymorphic Bond Operator](molecular-chemistry.md#the-polymorphic-bond-operator) - ---- - -*Consolidated: 2025-12-23* diff --git a/docs/session-communication.md b/docs/session-communication.md deleted file mode 100644 index 430666a4..00000000 --- a/docs/session-communication.md +++ /dev/null @@ -1,126 +0,0 @@ -# Session Communication: Nudge and Peek - -Gas Town agents communicate with Claude Code sessions through **two canonical commands**: - -| Command | Direction | Purpose | -|---------|-----------|---------| -| `gt nudge` | You → Agent | Send a message reliably | -| `gt peek` | Agent → You | Read recent output | - -## Why Not Raw tmux? - -**tmux send-keys is unreliable for Claude Code sessions.** - -The problem: When you send text followed by Enter using `tmux send-keys "message" Enter`, -the Enter key often arrives before the paste completes. Claude receives a truncated -message or the Enter appends to the previous line. - -This is a race condition in tmux's input handling. It's not a bug - tmux wasn't -designed for pasting multi-line content to interactive AI sessions. - -### The Reliable Pattern - -`gt nudge` uses a tested, reliable pattern: - -```go -// 1. Send text in literal mode (handles special characters) -tmux send-keys -t session -l "message" - -// 2. Wait 500ms for paste to complete -time.Sleep(500ms) - -// 3. Send Enter as separate command -tmux send-keys -t session Enter -``` - -**Never use raw tmux send-keys for agent sessions.** Always use `gt nudge`. - -## Command Reference - -### gt nudge - -Send a message to a polecat's Claude session: - -```bash -gt nudge - -# Examples -gt nudge gastown/furiosa "Check your mail and start working" -gt nudge gastown/alpha "What's your status?" -gt nudge gastown/beta "Stop what you're doing and read gt-xyz" -``` - -### gt peek - -View recent output from a polecat's session: - -```bash -gt peek [lines] - -# Examples -gt peek gastown/furiosa # Last 100 lines (default) -gt peek gastown/furiosa 50 # Last 50 lines -gt peek gastown/furiosa -n 200 # Last 200 lines -``` - -## Common Patterns - -### Check on a polecat - -```bash -# See what they're doing -gt peek gastown/furiosa - -# Ask for status -gt nudge gastown/furiosa "What's your current status?" -``` - -### Redirect a polecat - -```bash -# Stop current work and pivot -gt nudge gastown/furiosa "Stop current task. New priority: read and act on gt-xyz" -``` - -### Wake up a stuck polecat - -```bash -# Sometimes agents get stuck waiting for input -gt nudge gastown/furiosa "Continue working" -``` - -### Batch check all polecats - -```bash -# Quick status check -for p in furiosa nux slit; do - echo "=== $p ===" - gt peek gastown/$p 20 -done -``` - -## For Template Authors - -When writing agent templates (deacon.md.tmpl, polecat.md.tmpl, etc.), include this guidance: - -```markdown -## Session Communication - -To send messages to other agents, use gt commands: -- `gt nudge ` - Send reliably -- `gt peek ` - Read output - -⚠️ NEVER use raw tmux send-keys - it's unreliable for Claude sessions. -``` - -## Implementation Details - -The nudge/peek commands wrap these underlying functions: - -| Command | Wrapper | Underlying | -|---------|---------|------------| -| `gt nudge` | `tmux.NudgeSession()` | literal send-keys + delay + Enter | -| `gt peek` | `session.Capture()` | tmux capture-pane | - -The 500ms delay in NudgeSession was determined empirically. Shorter delays fail -intermittently; longer delays work but slow down communication unnecessarily. diff --git a/docs/session-lifecycle.md b/docs/session-lifecycle.md deleted file mode 100644 index 650e53e6..00000000 --- a/docs/session-lifecycle.md +++ /dev/null @@ -1,268 +0,0 @@ -# Session Lifecycle: Context Cycling in Gas Town - -> **Status**: Foundational Architecture -> **See also**: [beads-data-plane.md](beads-data-plane.md), [propulsion-principle.md](propulsion-principle.md) - -## Overview - -Gas Town agents have persistent identities, but their sessions are ephemeral. This is the -"cattle not pets" model from Kubernetes: the agent (Mayor, Witness, polecat Toast) is -the persistent identity; the Claude Code session running that agent is disposable cattle -that can be killed and respawned at any time. - -Work persists in beads. Sessions come and go. This document explains the unified model -for session lifecycle across all roles. - -## The Single-Bond Principle - -> A bead should be achievable in a single typical Claude Code session. - -In molecular chemistry, a single bond connects exactly two atoms. In Gas Town, a single -session should complete exactly one bead. This is the atomic unit of efficient work. - -**Why this matters:** -- Clean execution: start session → do work → cycle -- No mid-work state to track outside beads -- Handoffs are clean boundaries, not messy interruptions -- The system stays efficient as it scales - -**The ceiling rises over time.** What fits in "one session" grows with model capability. -Opus 4.5 can handle larger beads than earlier models. But the principle remains: if a -bead won't fit, decompose it. - -### Violating the Principle - -You can violate the Single-Bond Principle. Gas Town is flexible. But violations trigger -the **Happy Path** or **Sad Path**: - -**Happy Path** (controlled): -1. Worker recognizes mid-work: "This won't fit in one session" -2. Choose: - - (a) Partial implementation → handoff with context notes → successor continues - - (b) Self-decompose: convert bead to epic, create sub-tasks, continue with first task -3. If scope grew substantially → notify Witness/Mayor/Human for rescheduling - -**Sad Path** (uncontrolled): -- Worker underestimates → context fills unexpectedly → compaction triggers -- Compaction is the least desirable handoff: sudden, no context notes, state may be unclear -- Successor must reconstruct context from beads and git state - -The Happy Path preserves continuity. The Sad Path forces recovery. - -## The Context Budget Model - -**N is not "sessions" or "rounds" - it's a proxy for context budget.** - -Every action consumes context: - -| Action Type | Context Cost | Examples | -|-------------|--------------|----------| -| Boring | ~1-5% | Health ping, empty inbox check, clean rebase | -| Interesting | 20-50%+ | Conflict resolution, debugging, implementation | - -The heuristics (N=20 for Deacon, N=1 for Polecat) estimate "when will we hit ~80% context?" -They're tuning parameters, not laws of physics. - -### Why "Interesting" Triggers Immediate Handoff - -An "interesting" event consumes context like a full bead: -- Reading conflict diffs -- Debugging test failures -- Making implementation decisions -- Writing substantial code - -After an interesting event, you've used your context budget. Fresh session handles -the next event better than a session at 80% capacity. - -## Unified Session Cycling Model - -All workers use the same model. Only the heuristic differs: - -``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ SESSION LIFECYCLE │ -│ │ -│ ┌──────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────┐ │ -│ │ Start │───▶│ Execute Step │───▶│ Check Budget│───▶│ Handoff │ │ -│ │ Session │ │ │ │ │ │ or Done │ │ -│ └──────────┘ └──────┬───────┘ └──────┬──────┘ └──────────┘ │ -│ │ │ │ -│ │ ┌──────────────┘ │ -│ │ │ │ -│ ▼ ▼ │ -│ Budget OK? ──Yes──▶ Loop to next step │ -│ │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -### Per-Role Heuristics - -| Role | N | Unit | "Interesting" Trigger | -|------|---|------|----------------------| -| **Polecat** | 1 | assignment | Every assignment is interesting | -| **Crew** | ∞ | human decides | Context full or human requests | -| **Deacon** | 20 | patrol rounds | Lifecycle request, remediation, escalation | -| **Witness** | 15 | polecat interactions | (interactions are the cost unit) | -| **Refinery** | 20 | simple MRs | Complex rebase, conflict resolution | - -**Polecat (N=1)**: Every assignment fills context with implementation details. There's -no "boring" polecat work - it's all interesting. Forced to cycle when work is complete -and merged. - -**Crew (N=∞)**: Human-managed. Cycle when it feels right, or when context fills naturally. - -**Patrol Workers (N=15-20)**: Routine checks are cheap. Batch many before cycling. But -any "interesting" event resets to immediate handoff. - -## Mid-Step Handoff - -Any worker can handoff mid-step. The molecule tracks state: - -``` -Step 3 of 7, context filling - │ - ▼ - gt handoff -s "Context notes" -m "Details..." - │ - ▼ - Session dies, successor spawns - │ - ▼ - gt mol status → sees step 3 open → continues -``` - -This is normal and supported. The Single-Bond Principle is aspirational, not mandatory. -Mid-step handoff is the Happy Path when a bead turns out larger than estimated. - -## Handoff Mechanics - -How handoff works depends on the role: - -### Polecats: Outer Ring Handoff - -Polecats can't self-respawn - they need cleanup (worktree removal). They go through -their Witness: - -``` -Polecat Witness - │ │ - │ gt done (or gt handoff) │ - │ │ - ▼ │ -Send LIFECYCLE mail ──────────────▶ │ - │ │ - │ (wait for termination) ▼ - │ Patrol inbox-check step - │ │ - │ ▼ - │ Process LIFECYCLE request - │ │ - ◀─────────────────────────────── Kill session, cleanup worktree -``` - -### Non-Polecats: Self-Respawn - -Crew, Mayor, Witness, Refinery, Deacon are persistent. They respawn themselves: - -``` -Agent - │ - │ gt handoff -s "Context" -m "Notes" - │ - ▼ -Send handoff mail to self (optional) - │ - ▼ -tmux respawn-pane -k (kills self, starts fresh claude) - │ - ▼ -New session runs gt prime, finds hook, continues -``` - -No outer ring needed - they just restart in place. - -## State Persistence - -What survives a handoff: - -| Persists | Where | Example | -|----------|-------|---------| -| Pinned molecule | Beads (rig) | What you're working on | -| Handoff mail | Beads (town) | Context notes for successor | -| Git commits | Git | Code changes | -| Issue state | Beads | Open/closed, assignee, etc. | - -What doesn't survive: - -| Lost | Why | -|------|-----| -| Claude context | Session dies | -| In-memory state | Process dies | -| Uncommitted changes | Not in git | -| Unflushed beads | Not synced | - -**Key insight**: The molecule is the source of truth for work. Handoff mail is -supplementary context. You could lose all handoff mail and still know what to -work on from your hook. - -## The Oversized Bead Protocol - -When you realize mid-work that a bead won't fit in one session: - -### Option A: Partial Implementation + Handoff - -1. Commit what you have (even if incomplete) -2. Update bead description with progress notes -3. `gt handoff -s "Partial impl" -m "Completed X, remaining: Y, Z"` -4. Successor continues from your checkpoint - -Best when: Work is linear, successor can pick up where you left off. - -### Option B: Self-Decomposition - -1. Convert current bead to epic (if not already) -2. Create sub-task beads for remaining work -3. Close or update original with "decomposed into X, Y, Z" -4. Continue with first sub-task - -Best when: Work has natural breakpoints, parallelization possible. - -### When to Escalate - -Notify Witness/Mayor/Human when: -- Scope grew >2x from original estimate -- Decomposition affects other scheduled work -- Blockers require external input -- You're unsure which option to choose - -```bash -# Escalate to Witness (polecat) -gt mail send /witness -s "Scope change: " -m "..." - -# Escalate to Mayor (cross-rig) -gt mail send mayor/ -s "Scope change: " -m "..." - -# Escalate to Human -gt mail send --human -s "Need input: " -m "..." -``` - -## Summary - -1. **Single-Bond Principle**: One bead ≈ one session (aspirational) -2. **Context Budget**: N-heuristics proxy for "when will context fill?" -3. **Unified Model**: All roles cycle the same way, different heuristics -4. **Mid-Step Handoff**: Normal and supported via persistent molecules -5. **Happy Path**: Recognize early, decompose or partial-handoff -6. **Sad Path**: Compaction - recover from beads + git state - -The system is designed for ephemeral sessions with persistent state. Embrace cycling. -Fresh context handles problems better than stale context at capacity. - ---- - -## Related Documents - -- [beads-data-plane.md](beads-data-plane.md) - How state persists in beads -- [propulsion-principle.md](propulsion-principle.md) - The "RUN IT" protocol -- [pinned-beads-design.md](pinned-beads-design.md) - Hook mechanics -- [wisp-architecture.md](wisp-architecture.md) - Ephemeral patrol molecules diff --git a/docs/sling-design.md b/docs/sling-design.md deleted file mode 100644 index 8cc26c4f..00000000 --- a/docs/sling-design.md +++ /dev/null @@ -1,193 +0,0 @@ -# Hook, Sling, Handoff: Work Assignment Commands - -> **Status**: Implemented -> **Updated**: 2024-12-24 -> **See also**: [molecular-chemistry.md](molecular-chemistry.md) for the full Rig → Cook → Run lifecycle - -## The Propulsion Principle (GUPP) - -The Gastown Universal Propulsion Principle is simple: - -> **If you find something on your hook, YOU RUN IT.** - -No decisions. No "should I?" Just ignition. Agents execute what's on their -hook rather than deciding what to do. - -``` -Hook has work → Work happens. -``` - -That's the whole engine. Everything else is plumbing. - -## The Command Menagerie - -Three commands for putting work on hooks, each with distinct semantics: - -| Command | Action | Context | Use Case | -|---------|--------|---------|----------| -| `gt hook ` | Attach only | Preserved | Assign work for later | -| `gt sling ` | Attach + run | Preserved | Kick off work immediately | -| `gt handoff ` | Attach + restart | Fresh | Restart with new context | - -### The Relationships - -``` -gt hook = pin (assign) -gt sling = hook + run now -gt handoff = hook + restart (fresh context via GUPP) -``` - -A hypothetical `gt run` lurks in the liminal UX space - it would be "start -working on the hooked item now" without the attach step. Currently implicit -in startup via GUPP. - -## gt hook: Durability Primitive - -```bash -gt hook [flags] -``` - -**What it does**: Attaches work to your hook. Nothing more. - -**When to use**: -- You want to assign work but chat with the agent first -- Setting up work before triggering execution -- Preparing for a handoff without immediate restart - -**Example**: -```bash -gt hook gt-abc # Attach issue -gt hook gt-abc -s "Context here" # With subject for later handoff mail -``` - -The hook provides **durability** - the agent can restart, compact, or hand off, -but until the hook is changed or closed, that agent owns the work. - -## gt sling: Hook and Run Now - -```bash -gt sling [target] [flags] -``` - -**What it does**: -1. Attaches bead to the hook (durability) -2. Injects a prompt to start working NOW - -**When to use**: -- You've been chatting with an agent and want to kick off a workflow -- You want to assign work to another agent that has useful context -- You (Overseer) want to start work then attend to another window -- Starting work without losing current conversation context - -**Examples**: -```bash -gt sling gt-abc # Hook and start on it now -gt sling gt-abc -s "Fix the bug" # With context subject -gt sling gt-abc crew # Sling to crew worker -gt sling gt-abc gastown/crew/max # Sling to specific agent -``` - -**Key distinction from handoff**: Sling preserves context. The agent doesn't -restart - they receive an injected prompt and begin working with their current -conversation history intact. - -## gt handoff: Hook and Restart - -```bash -gt handoff [bead-or-role] [flags] -``` - -**What it does**: -1. If bead provided: attaches to hook first -2. Restarts the session (respawns pane) -3. New session wakes, finds hook, runs via GUPP - -**When to use**: -- Context has become too large or stale -- You want a fresh session but with work continuity -- Handing off your own session before context limits -- Triggering restart on another agent's session - -**Examples**: -```bash -gt handoff # Just restart (uses existing hook) -gt handoff gt-abc # Hook bead, then restart -gt handoff gt-abc -s "Fix it" # Hook with context, then restart -gt handoff -s "Context" -m "Notes" # Restart with handoff mail -gt handoff crew # Hand off crew session -gt handoff mayor # Hand off mayor session -``` - -**Interaction with roles**: The optional argument is polymorphic: -- If it looks like a bead ID (prefix `gt-`, `hq-`, `bd-`): hooks it -- Otherwise: treats it as a role to hand off - -## Agent Lifecycle with Hooks - -``` -┌────────────────────────────────────────────────────────────────┐ -│ Agent Hook Lifecycle │ -├────────────────────────────────────────────────────────────────┤ -│ │ -│ STARTUP (GUPP) │ -│ ┌─────────────────┐ │ -│ │ gt mol status │ → hook has work? → RUN IT │ -│ └─────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────┐ │ -│ │ Work on it │ ← agent executes molecule/bead │ -│ └─────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────┐ │ -│ │ Complete/Exit │ → close hook, check for next │ -│ └─────────────────┘ │ -│ │ -│ REASSIGNMENT (sling) │ -│ ┌─────────────────┐ │ -│ │ gt sling │ → hook updates, prompt injected │ -│ └─────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────┐ │ -│ │ Starts working │ ← preserves context │ -│ └─────────────────┘ │ -│ │ -│ CONTEXT REFRESH (handoff) │ -│ ┌─────────────────┐ │ -│ │gt handoff │ → hook updates, session restarts │ -│ └─────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────┐ │ -│ │ Fresh context │ ← GUPP kicks in on startup │ -│ └─────────────────┘ │ -│ │ -└────────────────────────────────────────────────────────────────┘ -``` - -## Command Quick Reference - -```bash -# Just assign, don't start -gt hook gt-xyz - -# Assign and start now (keep context) -gt sling gt-xyz -gt sling gt-xyz crew # To another agent - -# Assign and restart (fresh context) -gt handoff gt-xyz -gt handoff # Just restart, use existing hook - -# Check what's on hook -gt mol status -``` - -## See Also - -- `docs/propulsion-principle.md` - GUPP design -- `docs/molecular-chemistry.md` - Full Rig/Cook/Run lifecycle (formulas → protos → mols/wisps) -- `docs/wisp-architecture.md` - Ephemeral wisps for patrol loops -- `internal/wisp/` - Hook storage implementation diff --git a/docs/vision.md b/docs/vision.md deleted file mode 100644 index 676f8ec0..00000000 --- a/docs/vision.md +++ /dev/null @@ -1,413 +0,0 @@ -# Gas Town: Ideas We're Exploring - -> **Status**: These are ideas we're experimenting with, not proven solutions. -> We're sharing them in case others find them useful or want to explore together. - -Gas Town is an experiment in multi-agent coordination. We use steam-age metaphors -to think about how work flows through a system of AI agents. - ---- - -## Idea 1: The Steam Engine Metaphor - -``` -Claude = Fire (the energy source) -Claude Code = Steam Engine (harnesses the fire) -Gas Town = Steam Train (coordinates engines on tracks) -Beads = Railroad Tracks (the persistent ledger of work) -``` - -The engine does work and generates steam. Gas Town coordinates many engines on -a shared network, routing work to the right engines, tracking outcomes, and -ensuring nothing is lost. - -| Component | Phase | Role | -|-----------|-------|------| -| **Proto** | Solid (crystal) | Frozen workflow templates | -| **Mol** | Liquid | Flowing durable work instances | -| **Wisp** | Vapor (gas) | Transient ephemeral traces | -| **Digest** | Distillate | Compressed permanent records | - ---- - -## Idea 2: Village vs Hierarchy - -We're exploring whether a "village" model works better than centralized monitoring. - -**The pattern we're trying to avoid:** -``` -Centralized Monitor → watches all workers → single point of failure - → fragile protocols → cascading failures -``` - -**The pattern we're experimenting with:** -``` -Every worker → understands the whole → can help any neighbor - → peek is encouraged → distributed awareness - → ant colony without murder → self-healing system -``` - -### Aspiration: Antifragility - -We're aiming for anti-fragility - a system that gets stronger from stress rather -than just surviving it. Whether we achieve this is an open question. - -Properties we're trying to build: - -- **Distributed awareness**: Every agent understands the system deeply -- **Mutual monitoring**: Any agent can peek at any other agent's health -- **Collective intervention**: If you see something stuck, you can help -- **No single point of failure**: The village survives individual failures -- **Organic healing**: Problems get fixed by whoever notices them first - -This is an ant colony, except the ants don't kill defective members - they help -them recover. Workers who crash are respawned. Workers who get stuck are nudged. -Workers who need help receive it. - -### Practical Implications - -1. **Every patrol includes neighbor-checking** - - Polecats peek at other polecats - - Witness peeks at Refinery - - Refinery peeks at Witness - - Everyone can peek at the Deacon - -2. **`gt peek` is universal vocabulary** - - Any agent can check any other agent's health - - Health states are shared vocabulary: idle, working, stuck, done - -3. **Exit state enums are teaching tools** - - COMPLETED, BLOCKED, REFACTOR, ESCALATE - - Every agent learns these - - When peeking neighbors, agents recognize states and can help - -4. **Mail is the nervous system** - - Asynchronous, persistent, auditable - - Survives crashes and restarts - - The village communicates through mail - ---- - -## Idea 3: Rig, Cook, Run - -Work in Gas Town flows through three phases: - -``` -RIG ────→ COOK ────→ RUN -``` - -| Phase | What Happens | Key Operator | -|-------|--------------|--------------| -| **Rig** | Compose formulas (source level) | extends, compose | -| **Cook** | Instantiate work (pour/wisp) | cook, pour, wisp | -| **Run** | Execute steps | Agent execution | - -**Rig** is source-level composition (formula YAML). -**Bond** is artifact-level composition (protos, mols, wisps). - -See [molecular-chemistry.md](molecular-chemistry.md) for the full specification. - -### The Three Phases of Matter - -Work artifacts exist in three phases: - -| Phase | Name | State | Behavior | -|-------|------|-------|----------| -| **Solid** | Proto | Frozen template | Crystallized, immutable, reusable | -| **Liquid** | Mol | Flowing instance | Dynamic, adapting, persistent | -| **Vapor** | Wisp | Ephemeral trace | Transient, dissipates, operational | - -### Phase Transition Operators - -``` - ┌─────────────┐ - │ PROTO │ - │ (solid) │ - └──────┬──────┘ - │ - ┌─────────┼─────────┐ - │ │ │ - pour wisp distill - │ │ ↑ - ▼ ▼ │ - ┌─────────┐ ┌─────────┐ │ - │ MOL │ │ WISP │ │ - │(liquid) │ │ (vapor) │ │ - └────┬────┘ └────┬────┘ │ - │ │ │ - squash squash │ - │ │ │ - ▼ ▼ │ - ┌─────────┐ ┌─────────┐ │ - │ DIGEST │ │evaporates│ │ - │(crystal)│ │ or burn │ │ - └─────────┘ └──────────┘ │ - │ │ - └───────────────────┘ - (experience crystallizes) -``` - -| Operator | From | To | Effect | -|----------|------|------|--------| -| `pour` | Proto | Mol | Instantiate as persistent liquid | -| `wisp` | Proto | Wisp | Instantiate as ephemeral vapor | -| `bond` | Any + Any | Compound | Polymorphic combination | -| `squash` | Mol/Wisp | Digest | Condense to permanent record | -| `burn` | Wisp | Nothing | Discard without record | -| `distill` | Mol | Proto | Extract reusable template | - -### The Polymorphic Bond Operator - -**Bond** is the artifact-level combiner (distinct from **rig**, which composes -source formulas). Bond adapts to its operands: - -| bond | Proto | Mol | Wisp | -|------|-------|-----|------| -| **Proto** | Compound Proto | Pour + attach | Wisp + attach | -| **Mol** | Pour + attach | Link | Link | -| **Wisp** | Wisp + attach | Link | Link | - -This enables patterns like: -- Patrol wisp discovers issue → bonds new work mol -- Feature work needs diagnostic → bonds vapor wisp -- Witness tracks polecats → bonds lease per polecat - ---- - -## Idea 4: Beads as Data Plane - -We use Beads (a separate project) as the persistence layer. It's Git + Issues -in one human-readable format. - -**Properties we liked:** -- **Git-backed**: Uses existing infrastructure -- **Human-readable**: You can read `.beads/` files directly -- **Portable**: No vendor lock-in - -See [github.com/steveyegge/beads](https://github.com/steveyegge/beads) for more. - -### Git as Persistence - -Everything persists through git: -- Issues are JSONL in `.beads/` -- Molecules are structured issues -- Mail is issues with labels - -This gives us offline-first operation and no infrastructure requirements. - -### Control Plane = Data Plane - -Gas Town uses Beads as both control plane and data plane: - -| Data Type | Beads Representation | -|-----------|---------------------| -| Work items | Issues (tasks, bugs, features) | -| Workflows | Molecules (type=molecule) | -| Messages | Mail beads (type=message) | -| Merge requests | Queue entries (type=merge-request) | -| Agent state | Status on assigned issues | - -The control state IS data in Beads. Agents read Beads to know what to do next. -There is no separate orchestrator - Beads IS the orchestrator. - ---- - -## Idea 5: Patrol Loops - -Gas Town runs on continuous monitoring loops called **patrols**. - -### Patrol Agents - -| Agent | Role | Patrol Focus | -|-------|------|--------------| -| **Deacon** | Town-level daemon | Health of all agents, plugin execution | -| **Witness** | Per-rig polecat monitor | Polecat lifecycle, nudging, cleanup | -| **Refinery** | Per-rig merge processor | Merge queue, validation, integration | - -### Patrol Wisps - -Patrol agents run ephemeral wisps for their cycles: -- Wisp starts at cycle begin -- Steps complete as work progresses -- Wisp squashes to digest at cycle end -- New wisp created for next cycle - -This prevents accumulation: patrol work is vapor that condenses to minimal -digests, not liquid that pools forever. - -### The Witness Polecat-Tracking Wisp - -The Witness maintains a rolling wisp with a **lease** per active polecat: - -``` -wisp-witness-patrol -├── lease: furiosa (boot → working → done) -├── lease: nux (working) -└── lease: slit (done, closed) -``` - -Each lease is a bonded vapor molecule tracking one polecat's lifecycle. -When a polecat exits, its lease closes. When all leases close, the wisp -squashes to a summary digest. - ---- - -## Idea 6: Propulsion Over Protocol - -We're experimenting with "pull-based" work where agents propel themselves -rather than waiting for explicit commands: - -1. **Check hook/pin** - What's attached to me? -2. **Find next step** - What's ready in my molecule? -3. **Execute** - Do the work -4. **Advance** - Close step, find next -5. **Exit properly** - One of four exit types - -The idea is pull-based work: the molecule provides instructions, the agent executes. - -### Hooks and Pins - -Agents have **hooks** where work hangs. Work gets **pinned** to hooks. - -``` -Agent (with hook) - └── pinned mol (or wisp) - ├── step 1 (done) - ├── step 2 (in_progress) - └── step 3 (pending) -``` - -This enables: -- **Crash recovery**: Agent restarts, reads pinned mol, continues -- **Context survival**: Mol state persists across sessions -- **Handoff**: New session reads predecessor's pinned work -- **Observability**: `bd hook` shows what an agent is working on - -### The Four Exits - -Every polecat converges on one of four exits: - -| Exit | Meaning | Action | -|------|---------|--------| -| **COMPLETED** | Work finished | Submit to merge queue | -| **BLOCKED** | External dependency | File blocker, defer, notify | -| **REFACTOR** | Work too large | Break down, defer rest | -| **ESCALATE** | Need human judgment | Document, mail human, defer | - -All exits pass through the exit-decision step. All exits end in request-shutdown. -The polecat never exits directly - it waits to be killed by the Witness. - ---- - -## Idea 7: Nondeterministic Idempotence - -An idea we're exploring for crash recovery: - -- **Deterministic structure**: Molecule defines exactly what steps exist -- **Nondeterministic execution**: Any worker can execute any ready step -- **Idempotent progress**: Completed steps stay completed - -``` -Worker A picks up "design" step -Worker A completes "design" -Worker A crashes mid-"implement" -Worker B restarts, queries ready work -Worker B sees "implement" is ready (design done, implement pending) -Worker B continues from exactly where A left off -``` - -No work is lost. No state is in memory. Any worker can continue any molecule. - ---- - -## The Agent Hierarchy - -### Overseer (Human) -- Sets strategy and priorities -- Reviews and approves output -- Handles escalations -- Operates the system - -### Mayor (AI - Town-wide) -- Dispatches work across rigs -- Coordinates cross-project dependencies -- Handles strategic decisions - -### Deacon (AI - Town-level daemon) -- Ensures patrol agents are running -- Executes maintenance plugins -- Handles lifecycle requests - -### Witness (AI - Per-rig) -- Manages polecat lifecycle -- Detects stuck workers -- Handles session cycling - -### Refinery (AI - Per-rig) -- Processes merge queue -- Reviews and integrates code -- Maintains branch hygiene - -### Polecat (AI - Ephemeral workers) -- Executes work molecules -- Files discovered issues -- Ephemeral - spawn, work, disappear - ---- - -## The Steam Train in Action - -Putting it all together: - -``` -1. Human files issue in Beads -2. Mayor dispatches: gt sling --issue -3. Polecat created with: - - Fresh worktree - - mol-polecat-work pinned to hook - - Work assignment in mail -4. Deacon/Witness notified: POLECAT_STARTED -5. Witness bonds lease to patrol wisp -6. Polecat: - - Reads polecat.md, orients - - Reads mail, gets assignment - - Executes mol-polecat-work steps - - Makes commits, runs tests - - Submits to merge queue - - Exits via request-shutdown -7. Witness: - - Receives SHUTDOWN mail - - Closes polecat's lease - - Kills session, cleans up -8. Refinery: - - Processes merge queue - - Rebases, tests, merges - - Pushes to main -9. Digest created: work outcome crystallized -10. Loop: new work, new polecats, new cycles -``` - -The flywheel spins. The village watches itself. The train keeps running. - ---- - -## What We're Building Toward - -We're interested in exploring AI as collaborator rather than just assistant. -Instead of AI suggesting code, what if AI could complete tasks while humans -review the outcomes? - -This is early-stage experimentation. Some of it works, much of it doesn't yet. -We're sharing it in case the ideas are useful to others. - ---- - -## Summary - -Gas Town explores: -- Multi-agent coordination via "molecules" of work -- Git-backed persistence via Beads -- Distributed monitoring via the "village" model -- Crash recovery via nondeterministic idempotence - -We don't know if these ideas will pan out. We invite you to explore with us. diff --git a/docs/wisp-architecture.md b/docs/wisp-architecture.md deleted file mode 100644 index 761fab61..00000000 --- a/docs/wisp-architecture.md +++ /dev/null @@ -1,139 +0,0 @@ -# Wisp Architecture: Simplified - -> Status: Updated December 2024 - Simplified from separate directory to flag-based - -## Overview - -**Wisps** are ephemeral issues - transient workflow state that should not be synced -to the shared repository. They're used for operational messages (like lifecycle mail) -and patrol cycle traces that would otherwise accumulate unbounded. - -## Core Principle - -**Wisps are regular issues with `Wisp: true` flag.** - -The old architecture used a separate `.beads-wisp/` directory. This was over-engineered. -The simplified approach: - -| Old | New | -|-----|-----| -| `.beads-wisp/issues.jsonl` | `.beads/issues.jsonl` with `Wisp: true` | -| Separate directory, git init, gitignore | Single database, filtered on sync | -| Complex dual-inbox routing | Simple flag check | - -## How It Works - -### Creating Wisps - -```bash -# Create an ephemeral issue -bd create --title "Patrol cycle" --wisp - -# Send ephemeral mail (automatically sets Wisp=true) -gt mail send --wisp -s "Lifecycle: spawn" -m "..." -``` - -### Sync Filtering - -When `bd sync` exports to JSONL for git: -- Issues with `Wisp: true` are **excluded** -- Only permanent issues are synced to remote -- No separate directory needed - -### Querying - -```bash -# List all issues (including wisps) -bd list - -# List only wisps -bd list --wisp - -# List only permanent issues -bd list --no-wisp -``` - -## Use Cases - -### Ephemeral Mail (Lifecycle Messages) - -Spawn notifications, session handoffs, and other operational messages that: -- Don't need to be synced to remote -- Would accumulate unbounded -- Have no long-term audit value - -```bash -gt mail send gastown/polecats/nux --wisp \ - -s "LIFECYCLE: spawn" \ - -m "Work on issue gt-abc" -``` - -### Patrol Cycle Traces - -Deacon, Witness, and Refinery run continuous loops. Each cycle would create -accumulating history. Wisps let them track cycle state without permanent records. - -### Hook Files - -Agent hook files (`hook-.json`) are stored in `.beads/` but are local-only -runtime state, not synced. These track what work is assigned to each agent for -restart-and-resume. - -## Wisp Garbage Collection - -Wisps can accumulate over time. The `wisp-gc` doctor check cleans up stale wisps: - -```bash -# Check for stale wisps -gt doctor -v - -# Auto-cleanup stale wisps (>1h old by default) -gt doctor --fix - -# Manual cleanup via bd -bd wisp gc # Preview what would be cleaned -bd wisp gc --force # Actually delete stale wisps -bd wisp gc --age 2h # Custom age threshold -``` - -The Deacon includes wisp-gc in its patrol cycle via `gt doctor --fix`. - -### What Gets Cleaned - -- Wisps older than the threshold (default 1 hour) -- Wisps not associated with active sessions -- Orphaned lifecycle mail - -### What's Preserved - -- Wisps with active sessions still running -- Wisps younger than the threshold -- Non-wisp issues (permanent records) - -## Decision Matrix - -| Question | Answer | Use | -|----------|--------|-----| -| Should this sync to remote? | No | Wisp | -| Is this operational/lifecycle? | Yes | Wisp | -| Would this accumulate unbounded? | Yes | Wisp | -| Does this need audit trail? | Yes | Regular issue | -| Might others need to see this? | Yes | Regular issue | - -## Migration from .beads-wisp/ - -The old `.beads-wisp/` directories can be deleted: - -```bash -# Remove legacy wisp directories -rm -rf ~/gt/.beads-wisp/ -rm -rf ~/gt/gastown/.beads-wisp/ -find ~/gt -type d -name '.beads-wisp' -exec rm -rf {} + -``` - -No migration needed - these contained transient data with no long-term value. - -## Related - -- [molecules.md](molecules.md) - Molecule system (wisps can be molecule instances) -- [architecture.md](architecture.md) - Overall Gas Town architecture diff --git a/docs/witness-patrol-design.md b/docs/witness-patrol-design.md deleted file mode 100644 index 35ef4bb7..00000000 --- a/docs/witness-patrol-design.md +++ /dev/null @@ -1,94 +0,0 @@ -# Witness Patrol Design - -> The Witness is the Pit Boss. Oversight, not implementation. - -## Core Responsibilities - -| Duty | Action | -|------|--------| -| Handle POLECAT_DONE | Create cleanup wisp, process later | -| Handle HELP requests | Assess, help or escalate to Mayor | -| Ensure refinery alive | Restart if needed | -| Survey workers | Detect stuck polecats, nudge or escalate | -| Process cleanups | Verify git clean, kill session, burn wisp | - -## Patrol Shape (Linear) - -``` -inbox-check → process-cleanups → check-refinery → survey-workers → context-check → loop -``` - -No dynamic arms. No fanout gates. Simple loop like Deacon. - -## Key Design Principles - -| Principle | Meaning | -|-----------|---------| -| **Discovery over tracking** | Observe reality each cycle, don't maintain state | -| **Events over state** | POLECAT_DONE triggers wisps, not queue updates | -| **Cleanup wisps as finalizers** | Pending cleanup = wisp exists | -| **Task tool for parallelism** | Subagents inspect polecats, not molecule arms | -| **Fresh judgment each cycle** | No persistent nudge counters | - -## Cleanup: The Finalizer Pattern - -``` -POLECAT_DONE arrives - ↓ -Create wisp: bd create --wisp --title "cleanup:" --labels cleanup - ↓ -(wisp exists = cleanup pending) - ↓ -Witness process-cleanups step: - - Verify: git status clean, no unpushed, issue closed - - Execute: gt session kill, worktree removed - - Burn wisp - ↓ -Failed? Leave wisp, retry next cycle -``` - -## Assessing Stuck Polecats - -With step-based restarts, polecats are either: -- **Working a step**: Active tool calls, progress -- **Starting a step**: Just respawned, reading hook -- **Stuck on a step**: No progress, same step for multiple cycles - -| Observation | Action | -|-------------|--------| -| Active tool calls | None | -| Just started step (<5 min) | None | -| Idle 5-15 min, same step | Gentle nudge | -| Idle 15+ min, same step | Direct nudge | -| Idle 30+ min despite nudges | Escalate to Mayor | -| Errors visible | Assess, help or escalate | -| Says "done" but no POLECAT_DONE | Nudge to signal completion | - -**No persistent nudge counts**. Each cycle: observe reality, make fresh judgment. - -"How long stuck on same step" is discoverable from beads timestamps. - -## Parallelism via Task Tool - -Inspect multiple polecats concurrently using subagents: - -```markdown -## survey-workers step - -For each polecat, launch Task tool subagent: -- Capture tmux output -- Assess state (working/idle/error/done) -- Check beads for step progress -- Decide and execute action - -Task tool handles parallelism. One subagent per polecat. -``` - -## Formula - -See `.beads/formulas/mol-witness-patrol.formula.toml` - -## Related - -- [polecat-lifecycle.md](polecat-lifecycle.md) - Step-based execution model -- [molecular-chemistry.md](molecular-chemistry.md) - MEOW stack