docs: Plugins ARE molecules - remove YAML/markdown format

- Updated molecules.md to reflect actual JSONL storage format
- Updated architecture.md molecule sections with ASCII diagrams
- Replaced markdown "## Step:" format with tree diagrams
- Changed plugin system from directory-based to molecule-based
- Plugins now use labels for metadata (tier, phase, role)
- Mol Mall distributes molecules.jsonl fragments

Related: gt-u818, gt-9za0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Steve Yegge
2025-12-23 05:15:13 -08:00
parent 91992c7582
commit a473a35fe9
2 changed files with 350 additions and 877 deletions

View File

@@ -1,691 +1,293 @@
# Molecules: Composable Workflow Templates
This document covers the molecule system in depth, including pluggable molecules
and the code-review molecule design.
This document covers the molecule system in depth.
For an overview, see [architecture.md](architecture.md#molecules-composable-workflow-templates).
## Core Concepts
A **molecule** is a crystallized workflow pattern stored as a beads issue. When
instantiated on a parent issue, it creates child beads forming a DAG of steps.
A **molecule** is a workflow template stored as a beads issue with `labels: ["template"]`.
When bonded, it creates child issues forming a DAG of steps.
```
Proto (template)
▼ bd mol bond
┌─────────────────┐
│ Mol (durable) │ ← .beads/ (synced, auditable)
│ or │
│ Wisp (ephemeral)│ ← .beads-wisp/ (gitignored)
└────────┬────────┘
┌─────┴─────┐
▼ ▼
bd mol burn bd mol squash
(no record) (creates digest)
```
| Concept | Description |
|---------|-------------|
| Molecule | Read-only workflow template (type=molecule in beads) |
| Step | Individual unit of work within a molecule |
| Bond | Dependency between steps (Needs: directive) |
| Instance | Concrete beads created when molecule is instantiated |
| Proto | Template molecule (is_template: true in beads) |
| Mol | Durable instance in .beads/ |
| Wisp | Ephemeral instance in .beads-wisp/ |
| Digest | Condensed completion record |
| Bond | Instantiate proto → mol/wisp |
## Two Molecule Types
## Molecule Storage
### Static Molecules
Molecules are standard beads issues stored in `molecules.jsonl`:
Steps are embedded in the molecule's description field:
```markdown
## Step: design
Think carefully about architecture...
## Step: implement
Write the code...
Needs: design
## Step: test
Run tests...
Needs: implement
```json
{
"id": "mol-engineer-in-box",
"title": "Engineer in Box: {{feature_name}}",
"description": "Full workflow from design to merge.\n\nVars:\n- {{feature_name}} - What to build",
"labels": ["template"],
"issue_type": "epic"
}
```
**Use case**: Well-defined, fixed workflows (engineer-in-box, polecat-work).
**Loaded from multiple sources** (later overrides earlier):
1. Built-in (embedded in bd binary)
2. Town-level: `~/gt/.beads/molecules.jsonl`
3. User-level: `~/.beads/molecules.jsonl`
4. Project-level: `.beads/molecules.jsonl`
## Variable Substitution
Molecules support `{{var}}` placeholders resolved at bond time:
**Commands**:
```bash
bd create --type=molecule --title="My Workflow" --description="..."
gt molecule instantiate mol-xyz --parent=issue-123
bd mol bond mol-engineer-in-box --var feature_name="user auth"
# Creates: "Engineer in Box: user auth"
```
### Pluggable Molecules
Variables work in:
- Title
- Description
- Child step titles/descriptions
Steps are discovered from directories. Each directory is a plugin:
## Steps as Children
Steps are hierarchical children of the molecule:
```
~/gt/molecules/code-review/
├── discovery/
│ ├── file-census/CLAUDE.md
│ └── dep-graph/CLAUDE.md
├── structural/
│ └── architecture-review/CLAUDE.md
└── tactical/
├── security-scan/CLAUDE.md
└── performance-review/CLAUDE.md
mol-engineer-in-box (epic)
├── mol-engineer-in-box.1 "Design"
├── mol-engineer-in-box.2 "Implement" Needs: .1
├── mol-engineer-in-box.3 "Review" Needs: .2
├── mol-engineer-in-box.4 "Test" Needs: .3
└── mol-engineer-in-box.5 "Submit" Needs: .4
```
**Use case**: Extensible workflows where dimensions can be added/removed.
**Commands**:
```bash
gt molecule instantiate code-review --parent=issue-123 --scope=src/
```
Dependencies are encoded in beads edges, not in step descriptions.
## Molecule CLI Reference
The `bd mol` commands manage the molecule lifecycle: bonding (instantiation), squashing
(completion with digest), and burning (abandonment).
### bd mol bond
Instantiate a proto molecule into a runnable molecule (Mol or Wisp).
Instantiate a proto molecule into a runnable mol or wisp.
```bash
bd mol bond <proto-id> [--wisp] [--assignee=<addr>]
bd mol bond <proto-id> [--wisp] [--var key=value...]
```
**Arguments:**
| Argument | Required | Description |
|----------|----------|-------------|
| `<proto-id>` | Yes | ID of the proto molecule to instantiate |
| `--wisp` | No | Create a Wisp (ephemeral) instead of Mol (durable) |
| `--assignee` | No | Address of the agent who will execute this molecule |
**Behavior:**
- **Default (Mol)**: Creates a durable molecule tracked in the main `.beads/` database.
Steps become permanent issues that survive indefinitely.
- **With --wisp**: Creates a wisp (transient molecule) in `.beads-wisp/`. Steps are
transient and will be cleaned up on squash or burn.
| Flag | Description |
|------|-------------|
| `--wisp` | Create ephemeral wisp instead of durable mol |
| `--var` | Variable substitution (repeatable) |
| `--ref` | Custom reference ID for the instance |
**Examples:**
```bash
# Create a durable Mol from engineer-in-box proto
bd mol bond mol-engineer-in-box
# Durable mol
bd mol bond mol-engineer-in-box --var feature_name="auth"
# Create a Wisp assigned to a polecat
bd mol bond mol-quick-fix --wisp --assignee=gastown/polecats/toast
# Ephemeral wisp for patrol
bd mol bond mol-witness-patrol --wisp
# Bond a proto with a specific template variant
bd mol bond mol-engineer-in-box --template=minimal
# Dynamic child with custom ref (Christmas Ornament pattern)
bd mol bond mol-polecat-arm $PATROL_ID --ref arm-ace --var polecat=ace
```
**Output:**
Returns the ID of the newly created molecule instance (e.g., `gt-xyz.exec-001`).
---
### bd mol squash
Complete a molecule and generate a digest (permanent summary record).
Complete a molecule and generate a permanent digest.
```bash
bd mol squash <mol-id> --summary='...'
```
**Arguments:**
The summary is agent-generated - the intelligence comes from the agent, not beads.
| Argument | Required | Description |
|----------|----------|-------------|
| `<mol-id>` | Yes | ID of the molecule instance to squash |
| `--summary` | Yes | Summary of what was accomplished (agent-generated) |
**Behavior:**
- **For Mol (durable)**: Creates a digest issue in the permanent beads database.
The digest contains the summary and links back to the original proto.
- **For Wisp (transient)**: Evaporates the wisp (deletes from `.beads-wisp/`)
and creates a digest in the permanent database. The execution trace is gone,
but the outcome is preserved.
**The summary is critical**: Agents generate the summary describing what was
accomplished. This becomes the permanent record. Beads is a tool - the intelligence
for summarization comes from the agent.
**Examples:**
```bash
# Squash a completed molecule with summary
bd mol squash gt-xyz.exec-001 --summary='Implemented user auth with JWT tokens. Added login/logout endpoints and middleware for protected routes.'
# Squash with multi-line summary (use quotes)
bd mol squash gt-xyz.exec-001 --summary='Fixed authentication bug.
Changes:
- Corrected token expiry calculation
- Added refresh token rotation
- Updated tests'
```
**Output:**
Returns the ID of the created digest (e.g., `gt-xyz.digest-001`).
---
**For Mol**: Creates digest in .beads/, original steps remain.
**For Wisp**: Evaporates wisp, creates digest. Execution trace gone, outcome preserved.
### bd mol burn
Abandon a molecule without completing it. No digest is created.
Abandon a molecule without record.
```bash
bd mol burn <mol-id> [--reason='...']
```
**Arguments:**
Use burn for:
- Routine patrol cycles (no audit needed)
- Failed experiments
- Cancelled work
| Argument | Required | Description |
|----------|----------|-------------|
| `<mol-id>` | Yes | ID of the molecule instance to burn |
| `--reason` | No | Explanation of why the molecule was abandoned |
Use squash for:
- Completed work
- Investigations with findings
- Anything worth recording
**Behavior:**
### bd mol list
- Discards all molecule state (steps, progress, artifacts)
- No digest is created - the molecule leaves no permanent record
- For Wisps: Simply deletes from `.beads-wisp/`
- For Mols: Marks as abandoned/closed without digest
**When to burn vs squash:**
| Situation | Use |
|-----------|-----|
| Work completed successfully | `squash` - preserve the outcome |
| Routine work, no audit needed | `burn` - no record necessary |
| Work abandoned/cancelled | `burn` - nothing to record |
| Experiment that failed | `burn` - don't clutter history |
| Investigation with findings | `squash` - findings are valuable |
**Examples:**
List available molecules:
```bash
# Burn a molecule that's no longer needed
bd mol burn gt-xyz.exec-001
# Burn with explanation
bd mol burn gt-xyz.exec-001 --reason='Superseded by different approach in gt-abc'
# Burn a failed experiment
bd mol burn gt-xyz.exec-001 --reason='Approach proved unworkable - see notes in PR #123'
bd mol list # All templates
bd mol list --label plugin # Plugin molecules only
```
---
## Plugins ARE Molecules
### Lifecycle Summary
Patrol plugins are molecules with specific labels:
```
Proto Molecule (template in catalog)
▼ bd mol bond
┌────────────────────┐
│ Mol (durable) or │
│ Wisp (ephemeral) │
└────────┬───────────┘
┌────────┴────────┐
▼ ▼
bd mol burn bd mol squash
(no record) (creates digest)
```json
{
"id": "mol-security-scan",
"title": "Security scan for {{polecat_name}}",
"description": "Check for vulnerabilities.\n\nVars: {{polecat_name}}, {{captured_output}}",
"labels": ["template", "plugin", "witness", "tier:haiku"],
"issue_type": "task"
}
```
**Key insight**: The molecule lifecycle maps to Gas Town's steam engine metaphor:
- **Proto** = Fuel (frozen template)
- **Mol/Wisp** = Steam (active execution)
- **Digest** = Distillate (condensed permanent record)
- **Burn** = Steam dissipates (no condensation)
- **Squash** = Steam condenses (captured as digest)
Label conventions:
- `plugin` - marks as bondable at hook points
- `witness` / `deacon` / `refinery` - which patrol uses it
- `tier:haiku` / `tier:sonnet` - model hint
## Plugin CLAUDE.md Format
**Execution in patrol:**
Each plugin directory contains a CLAUDE.md with optional frontmatter:
```markdown
---
phase: tactical
needs: [structural-complete]
tier: sonnet
---
# Security Scan
## Objective
Identify security vulnerabilities in the target code.
## Focus Areas
- OWASP Top 10 vulnerabilities
- Injection attacks (SQL, command, LDAP)
- Authentication/authorization bypasses
- Hardcoded secrets
- Insecure deserialization
## Output
For each finding, create a bead with:
- Clear title describing the vulnerability
- File path and line numbers
- Severity (P0-P4)
- Suggested remediation
Tag findings with `label: security`.
```bash
# plugin-run step bonds registered plugins:
bd mol bond mol-security-scan $PATROL_WISP \
--ref security-{{polecat_name}} \
--var polecat_name=ace \
--var captured_output="$OUTPUT"
```
### Frontmatter Fields
## The Christmas Ornament Pattern
| Field | Type | Description |
|-------|------|-------------|
| `phase` | string | Execution phase: discovery, structural, tactical, synthesis |
| `needs` | list | Step references that must complete first |
| `tier` | string | Model hint: haiku, sonnet, opus |
Dynamic bonding creates tree structures at runtime:
### Phase Semantics
```
★ mol-witness-patrol
/|\
┌─────┘ │ └─────┐
PREFLIGHT │ CLEANUP
│ │ │
┌───┴───┐ │ ┌───┴───┐
│inbox │ │ │aggreg │
│load │ │ │summary│
└───────┘ │ └───────┘
┌─────────┼─────────┐
│ │ │
● ● ● mol-polecat-arm (dynamic)
ace nux toast
│ │ │
┌──┴──┐ ┌──┴──┐ ┌──┴──┐
│steps│ │steps│ │steps│
└──┬──┘ └──┬──┘ └──┬──┘
│ │ │
└─────────┴─────────┘
⬣ base
```
| Phase | Blocks | Parallelism | Purpose |
|-------|--------|-------------|---------|
| discovery | nothing | full | Inventory, gather data |
| structural | discovery | sequential | Big picture analysis |
| tactical | structural | per-component | Detailed review |
| synthesis | tactical | single | Aggregate results |
**Key primitives:**
- `bd mol bond ... --ref` - creates named children
- `WaitsFor: all-children` - fanout gate
- Arms execute in parallel
## Mol Mall
Distribution through molecule marketplace:
```bash
# Install from registry
bd mol install mol-security-scan
# Updates ~/.beads/molecules.jsonl
```
Mol Mall serves `molecules.jsonl` fragments. Installation appends to your catalog.
## Code Review Molecule
The code-review molecule is the reference implementation for pluggable molecules.
### Directory Structure
The code-review molecule is a pluggable workflow:
```
~/gt/molecules/code-review/
├── README.md # Molecule overview
mol-code-review
├── discovery (parallel)
│ ├── file-census
│ ├── dep-graph
│ └── coverage-map
├── discovery/ # Phase 1: Parallel scouts
│ ├── file-census/
│ └── CLAUDE.md # Inventory: sizes, ages, churn
│ ├── dep-graph/
│ │ └── CLAUDE.md # Dependencies, cycles, inversions
│ ├── coverage-map/
│ │ └── CLAUDE.md # Test coverage, dead code
│ └── duplication-scan/
│ └── CLAUDE.md # Near-duplicates, copy-paste
├── structural (sequential)
│ ├── architecture-review
│ └── abstraction-analysis
├── structural/ # Phase 2: Sequential for coherence
│ ├── architecture-review/
│ └── CLAUDE.md # Structure vs domain alignment
── abstraction-analysis/
│ │ └── CLAUDE.md # Wrong-layer wrangling
│ └── consolidation-planner/
│ └── CLAUDE.md # What should be unified
├── tactical (parallel per component)
│ ├── security-scan
├── performance-review
── complexity-analysis
── tactical/ # Phase 3: Parallel per hotspot
── security-scan/
│ │ └── CLAUDE.md # OWASP, injection, auth
│ ├── performance-review/
│ │ └── CLAUDE.md # N+1, caching, memory
│ ├── complexity-analysis/
│ │ └── CLAUDE.md # Cyclomatic, nesting
│ ├── test-gaps/
│ │ └── CLAUDE.md # Untested paths, edge cases
│ └── elegance-review/
│ └── CLAUDE.md # Magic numbers, naming
└── synthesis/ # Phase 4: Single coordinator
└── aggregate/
└── CLAUDE.md # Dedupe, prioritize, sequence
── synthesis (single)
── aggregate
```
### Discovery Phase Plugins
Each dimension is a molecule that can be:
- Swapped for alternatives from Mol Mall
- Customized by forking and installing your version
- Disabled by not bonding it
#### file-census
**Purpose**: Build inventory of what we're reviewing.
**Output**:
- Total files, lines, and size
- Files by age (old = potential legacy)
- Files by churn (high churn = hotspots)
- Largest files (candidates for splitting)
#### dep-graph
**Purpose**: Map dependencies and structure.
**Output**:
- Dependency graph (imports, requires)
- Circular dependencies
- Orphaned code (unreachable)
- Inverted dependencies (high-level depending on low-level)
#### coverage-map
**Purpose**: Understand test coverage.
**Output**:
- Overall coverage percentage
- Untested files/functions
- Coverage by component
- Dead code (never executed)
#### duplication-scan
**Purpose**: Find duplicated logic.
**Output**:
- Near-duplicate files
- Copy-paste code blocks
- Redundant implementations of same concept
### Structural Phase Plugins
#### architecture-review
**Purpose**: Assess high-level structure.
**Questions**:
- Does directory structure match domain concepts?
- Are boundaries clean between components?
- Is there a clear layering strategy?
- Are cross-cutting concerns (logging, auth) handled consistently?
**Output**: Structural findings as beads, with refactoring recommendations.
#### abstraction-analysis
**Purpose**: Find missing or wrong abstractions.
**Signs of problems**:
- Same boilerplate repeated
- Business logic mixed with infrastructure
- Leaky abstractions (implementation details exposed)
- Primitive obsession (should be domain types)
**Output**: Abstraction issues as beads.
#### consolidation-planner
**Purpose**: Identify what should be unified.
**Looks for**:
- Multiple implementations of same concept
- Similar code in different places
- Parallel hierarchies
- Scattered handling of same concern
**Output**: Consolidation recommendations as beads.
### Tactical Phase Plugins
These run in parallel, each agent reviewing assigned files/components.
#### security-scan
**Focus**:
- OWASP Top 10
- Injection vulnerabilities
- Authentication/authorization issues
- Secrets in code
- Insecure configurations
#### performance-review
**Focus**:
- N+1 queries
- Missing caching opportunities
- Memory leaks
- Unnecessary computation
- Blocking operations in hot paths
#### complexity-analysis
**Focus**:
- Cyclomatic complexity > 10
- Deep nesting (> 4 levels)
- Long functions (> 50 lines)
- God classes/files
- Complex conditionals
#### test-gaps
**Focus**:
- Untested public APIs
- Missing edge cases
- No error path testing
- Brittle tests (mock-heavy, order-dependent)
#### elegance-review
**Focus**:
- Magic numbers/strings
- Unclear naming
- Inconsistent style
- Missing documentation for complex logic
- Overly clever code
### Synthesis Phase
#### aggregate
**Purpose**: Combine all findings into actionable backlog.
**Tasks**:
1. Deduplicate similar findings
2. Group related issues
3. Establish fix dependencies (fix X before Y)
4. Prioritize by impact
5. Sequence for efficient fixing
**Output**: Prioritized backlog ready for swarming.
## Implementation Plan
### Phase 1: Pluggable Molecule Infrastructure
1. **Directory scanner** (`internal/molecule/scanner.go`)
- Scan molecule directories for plugins
- Parse CLAUDE.md frontmatter
- Build plugin registry
2. **DAG builder** (`internal/molecule/dag.go`)
- Assemble dependency graph from plugins
- Respect phase ordering
- Validate no cycles
3. **Instantiation** (`internal/molecule/instantiate.go`)
- Create beads for each step
- Wire dependencies
- Support scope parameter
### Phase 2: Code Review Molecule
1. **Plugin directory structure**
- Create ~/gt/molecules/code-review/
- Write CLAUDE.md for each dimension
2. **Discovery plugins** (4)
- file-census, dep-graph, coverage-map, duplication-scan
3. **Structural plugins** (3)
- architecture-review, abstraction-analysis, consolidation-planner
4. **Tactical plugins** (5)
- security-scan, performance-review, complexity-analysis, test-gaps, elegance-review
5. **Synthesis plugin** (1)
- aggregate
### Phase 3: CLI Integration
1. **gt molecule scan** - Show discovered plugins
2. **gt molecule validate** - Validate plugin structure
3. **gt molecule instantiate** - Create beads from plugins
4. **gt review** - Convenience wrapper for code-review molecule
## Usage Examples
### Basic Code Review
**Usage:**
```bash
# Run full code review on project
gt molecule instantiate code-review --parent=gt-review-001
# Check what's ready to work
bd ready
# Swarm it
gt swarm --parent=gt-review-001
bd mol bond mol-code-review --var scope="src/"
```
### Scoped Review
```bash
# Review only src/auth/
gt molecule instantiate code-review --parent=gt-review-002 --scope=src/auth
# Review only tactical dimensions
gt molecule instantiate code-review --parent=gt-review-003 --phases=tactical
```
### Adding a Custom Dimension
```bash
# Create plugin directory
mkdir -p ~/gt/molecules/code-review/tactical/accessibility-review
# Add CLAUDE.md
cat > ~/gt/molecules/code-review/tactical/accessibility-review/CLAUDE.md << 'EOF'
---
phase: tactical
needs: [structural-complete]
tier: sonnet
---
# Accessibility Review
Check for WCAG 2.1 compliance issues...
EOF
# Now it's automatically included in code-review
gt molecule scan code-review
```
### Iteration
```bash
# First review pass
gt molecule instantiate code-review --parent=gt-review-001
# ... fix issues ...
# Second pass (fewer findings expected)
gt molecule instantiate code-review --parent=gt-review-002
# ... fix remaining issues ...
# Third pass (should be at noise floor)
gt molecule instantiate code-review --parent=gt-review-003
```
## Beads Generated by Reviews
Each review step generates findings as beads:
## Lifecycle Summary
```
gt-sec-001 SQL injection in login() type=bug priority=1 label=security
gt-sec-002 Missing CSRF protection type=bug priority=2 label=security
gt-perf-001 N+1 query in user list type=bug priority=2 label=performance
gt-arch-001 Auth logic in controller type=task priority=3 label=refactor
┌─────────────┐
│ Proto │ Template in molecules.jsonl
│ (frozen) │ labels: ["template"]
└──────┬──────┘
▼ bd mol bond [--wisp]
┌─────────────┐
│ Mol/Wisp │ Active execution
│ (flowing) │ Mol: .beads/ | Wisp: .beads-wisp/
└──────┬──────┘
┌────┴────┐
▼ ▼
burn squash
│ │
▼ ▼
(gone) Digest
(permanent)
```
Findings link back to the review:
```
discovered-from: gt-review-001.tactical-security-scan
```
This enables querying: "What did the security scan find?"
## Feed the Beast Pattern
Code review is a **work generator**:
```
Low on beads?
gt molecule instantiate code-review
Generates 50-200 fix beads
Prioritize and swarm
Codebase improves overnight
Repeat weekly
```
## Cross-Project Dependencies and Parking
Molecules can hit external dependencies that block progress. See
[cross-project-deps.md](cross-project-deps.md) for the full design.
### Parking a Molecule
When a molecule step depends on a capability from another project:
```bash
# Polecat discovers external dependency not satisfied
gt park --step=gt-mol.3 --waiting="beads:mol-run-assignee"
```
The molecule enters a **parked** state (derived from: in_progress + no assignee + blocked step).
### Resuming a Parked Molecule
When the external dependency is satisfied:
```bash
# Manual resume
gt spawn --continue gt-mol-root
# Or automated via Deacon patrol (future)
```
### Parked State
"Parked" is not a new status - it's derived from existing fields:
- Molecule status: `in_progress`
- Molecule assignee: `null` (no polecat owns it)
- Has step with unsatisfied `blocked_by: external:...`
Query parked molecules:
```bash
bd list --status=in_progress --no-assignee --type=molecule
```
## Future Extensions
### Custom Molecule Types
Beyond code-review, pluggable molecules could support:
- **migration-analysis**: Database migrations, API versioning
- **onboarding-review**: New hire documentation gaps
- **compliance-audit**: Regulatory requirements check
- **dependency-audit**: Outdated/vulnerable dependencies
### Scheduled Reviews
```yaml
# In rig config
scheduled_molecules:
- molecule: code-review
scope: "**/*.go"
schedule: "0 0 * * 0" # Weekly Sunday midnight
priority: 3
```
### Review Trends
Track findings over time:
```bash
gt review history --molecule=code-review
# Shows: findings per run, categories, fix rate
```
**Steam engine metaphor:**
- Proto = Fuel (potential energy)
- Mol/Wisp = Steam (kinetic energy)
- Digest = Distillate (crystallized work)
- Burn = Steam dissipates
- Squash = Steam condenses