# Merge Queue Design The merge queue coordinates landing completed work. MRs are ephemeral wisps (not synced beads), and polecat branches stay local (never pushed to origin). **Key insight**: Git is already a ledger. Beads track durable state (issues). MRs are transient operational state - perfect for wisps. ## Architecture (Current) ``` ┌─────────────────────────────────────────────────────────────────┐ │ LOCAL MERGE QUEUE │ │ │ │ Polecat worktree ──commit──► local branch (polecat/nux) │ │ │ │ │ │ │ │ (same .git) │ │ ▼ ▼ │ │ .beads-wisp/mq/mr-xxx.json Refinery worktree │ │ │ │ │ │ └──────────────────────────┘ │ │ │ │ │ Refinery reads MR, merges branch │ │ │ │ │ ▼ │ │ git push origin main │ │ │ │ │ Delete local branch │ │ Delete MR wisp file │ └─────────────────────────────────────────────────────────────────┘ ``` **Key points:** - Refinery is a worktree of the same git repo as polecats (shared .git) - Polecat branches are local only - never pushed to origin - MRs stored in `.beads-wisp/mq/` (ephemeral, not synced) - Only `main` branch gets pushed to origin after merge - Source issues tracked in beads (durable), closed after merge ## Overview ``` ┌─────────────────────────────────────────────────────────────────┐ │ MERGE QUEUE │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ MR #1 │→ │ MR #2 │→ │ MR #3 │→ │ MR #4 │ │ │ │ (ready) │ │(blocked) │ │ (ready) │ │(blocked) │ │ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │ ↓ ↓ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ ENGINEER (processes queue) │ │ │ │ 1. bd ready --type=merge-request │ │ │ │ 2. Process in priority order │ │ │ │ 3. Merge, test, close or reject │ │ │ └─────────────────────────────────────────────────────┘ │ │ ↓ │ │ ┌──────────┐ │ │ │ main │ │ │ └──────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` ## Merge Request Schema A merge request is a JSON file in `.beads-wisp/mq/`: ```json // .beads-wisp/mq/mr-1703372400-a1b2c3d4.json { "id": "mr-1703372400-a1b2c3d4", "branch": "polecat/nux", "target": "main", "source_issue": "gt-xyz", "worker": "nux", "rig": "gastown", "title": "Merge: gt-xyz", "priority": 1, "created_at": "2025-12-23T20:00:00Z" } ``` ### ID Convention MR IDs follow the pattern: `mr--` Example: `mr-1703372400-a1b2c3d4` These are ephemeral - deleted after merge. Source issues in beads provide the durable record. ### Creating Merge Requests Workers submit to the queue via: ```bash # Worker signals work is ready (preferred) gt done # Auto-detects branch, issue, creates MR # Or explicit submission gt mq submit # Auto-detects branch, issue, worker gt mq submit --issue gt-xyz # Explicit issue # Under the hood, this writes to .beads-wisp/mq/: # - Creates mr--.json # - No beads issue created (MRs are ephemeral wisps) # - Branch stays local (never pushed to origin) ``` ## Queue Ordering The queue is ordered by: 1. **Dependencies first** - If MR-B depends on MR-A, A merges first 2. **Priority** - P0 before P1 before P2 3. **Age** - Older requests before newer (FIFO within priority) ### Dependency-Based Ordering When workers complete related work, dependencies ensure correct order: ``` gt-epic (epic) ├── gt-epic.1 (task) → gt-mr-001 (merge request) ├── gt-epic.2 (task) → gt-mr-002 (depends on gt-mr-001) └── gt-epic.3 (task) → gt-mr-003 (depends on gt-mr-002) ``` The Engineer queries `bd ready --type=merge-request` to find MRs with no unmerged dependencies. ### Integration Branches For batch work on an epic, use an integration branch: ``` main │ ├──────────────────────────┐ │ │ integration/gt-epic (other work) │ ┌──────────┼──────────┐ │ │ │ MR #1 MR #2 MR #3 ``` Integration branch workflow: 1. Create integration branch from main: `integration/gt-epic` 2. MRs target the integration branch, not main 3. After all epic work merges, integration branch merges to main 4. Single PR for epic = easier review, atomic landing ```bash # Create integration branch for epic gt mq integration create gt-epic # MRs auto-target integration branch gt mq submit --epic gt-epic # Targets integration/gt-epic # Land entire epic to main gt mq integration land gt-epic ``` ## Engineer Processing Loop The Engineer (formerly Refinery) processes the merge queue continuously: ``` ┌─────────────────────────────────────────────────────────────┐ │ ENGINEER LOOP │ │ │ │ while true: │ │ 1. ready_mrs = bd ready --type=merge-request │ │ │ │ 2. if ready_mrs.empty(): │ │ sleep(poll_interval) │ │ continue │ │ │ │ 3. mr = ready_mrs.first() # Highest priority, oldest │ │ │ │ 4. bd update mr.id --status=in_progress │ │ │ │ 5. result = process_merge(mr) │ │ │ │ 6. if result.success: │ │ bd close mr.id --reason="merged: {sha}" │ │ else: │ │ handle_failure(mr, result) │ │ │ │ 7. update_source_issue(mr) │ └─────────────────────────────────────────────────────────────┘ ``` ### Process Merge Steps ```python def process_merge(mr): # 1. Branch is already local (shared .git with polecats) # No fetch needed - refinery worktree sees polecat branches directly # 2. Check for conflicts with target conflicts = git_check_conflicts(mr.branch, mr.target) if conflicts: return Failure(reason="conflict", files=conflicts) # 3. Merge to local target git checkout {mr.target} git merge {mr.branch} --no-ff -m "Merge {mr.branch}: {mr.title}" # 4. Run tests (configurable) if config.run_tests: result = run_tests() if result.failed: git reset --hard HEAD~1 # Undo merge return Failure(reason="tests_failed", output=result.output) # 5. Push to origin (only main goes to origin) git push origin {mr.target} # 6. Clean up source branch (local delete only) if config.delete_merged_branches: git branch -D {mr.branch} # Local delete, not remote # 7. Remove MR wisp file os.remove(.beads-wisp/mq/{mr.id}.json) return Success(merge_commit=git_rev_parse("HEAD")) ``` ### Handling Failures | Failure | Action | |---------|--------| | **Conflict** | Assign back to worker, add `needs-rebase` label | | **Tests fail** | Assign back to worker, add `needs-fix` label | | **Build fail** | Assign back to worker, add `needs-fix` label | | **Flaky test** | Retry once, then assign back | | **Infra issue** | Retry with backoff, escalate if persistent | ```bash # On conflict, Engineer does: bd update gt-mr-xxx --assignee=Nux --labels=needs-rebase gt send gastown/Nux -s "Rebase needed: gt-mr-xxx" \ -m "Your branch conflicts with main. Please rebase and resubmit." ``` ### Conflict Resolution Strategies 1. **Assign back to worker** (default) - Worker rebases and resubmits 2. **Auto-rebase** (configurable) - Engineer attempts `git rebase` automatically 3. **Semantic merge** (future) - AI-assisted conflict resolution ```yaml # rig config.json { "merge_queue": { "on_conflict": "assign_back", # or "auto_rebase" or "semantic" "run_tests": true, "delete_merged_branches": true, "retry_flaky_tests": 1 } } ``` ## CLI Commands ### gt mq (merge queue) ```bash # Submit work to queue gt mq submit # Auto-detect from current branch gt mq submit --issue gt-xyz # Explicit issue gt mq submit --epic gt-epic # Target integration branch # View queue gt mq list # Show all pending MRs gt mq list --ready # Show only ready-to-merge gt mq list --mine # Show MRs for my work gt mq status gt-mr-xxx # Detailed MR status # Integration branches gt mq integration create gt-epic # Create integration/gt-epic gt mq integration land gt-epic # Merge integration to main gt mq integration status gt-epic # Show integration branch status # Admin/debug gt mq retry gt-mr-xxx # Retry a failed MR gt mq reject gt-mr-xxx --reason "..." # Reject an MR gt mq reorder gt-mr-xxx --after gt-mr-yyy # Manual reorder ``` ### Command Details #### gt mq submit ```bash gt mq submit [--branch BRANCH] [--issue ISSUE] [--epic EPIC] # Auto-detection logic: # 1. Branch: current git branch # 2. Issue: parse from branch name (polecat/Nux/gt-xyz → gt-xyz) # 3. Epic: if issue has parent epic, offer integration branch # Creates merge-request bead and prints MR ID ``` #### gt mq list ```bash gt mq list [--ready] [--status STATUS] [--worker WORKER] # Output: # ID STATUS PRIORITY BRANCH WORKER AGE # gt-mr-001 ready P0 polecat/Nux/gt-xyz Nux 5m # gt-mr-002 in_progress P1 polecat/Toast/gt-abc Toast 12m # gt-mr-003 blocked P1 polecat/Capable/gt-def Capable 8m # (waiting on gt-mr-001) ``` ## Beads Query Patterns The merge queue is just queries over beads: ```bash # Ready to merge (no blockers, not in progress) bd ready --type=merge-request # All open MRs bd list --type=merge-request --status=open # MRs for a specific epic (via labels or source_issue parent) bd list --type=merge-request --label=epic:gt-xyz # Recently merged bd list --type=merge-request --status=closed --since=1d # MRs by worker bd list --type=merge-request --assignee=Nux ``` ## State Machine ``` ┌─────────────┐ │ CREATED │ │ (open) │ └──────┬──────┘ │ ┌──────▼──────┐ ┌───────│ READY │───────┐ │ │ (open) │ │ │ └──────┬──────┘ │ │ │ │ blocked by ┌─────▼─────┐ rejected dependency │ PROCESSING│ (manual) │ │(in_progress) │ │ └─────┬─────┘ │ │ │ │ │ ┌────────┴────────┐ │ │ │ │ │ │ success failure │ │ │ │ │ ▼ ▼ │ │ ┌───────┐ ┌─────────┐│ │ │MERGED │ │ FAILED ││ │ │(closed) │ (open) ││ │ └───────┘ └────┬────┘│ │ │ │ │ resubmit │ │ │ │ └─────────────────┴─────────┘ ``` ## Audit and Observability The merge queue creates a complete audit trail for all integrated work: | MQ Event | Record Created | |----------|----------------| | Merge request submitted | Work completion claim with author | | Tests pass | Quality verification record | | Refinery approves | Validation with reviewer attribution | | Merge commit | Immutable integration record | | Rejection | Feedback record with reason | Every merge creates an immutable record: - Who did the work (author attribution) - Who validated it (Refinery attestation) - When it landed (timestamp) - What changed (commit diff) This enables full work attribution and quality tracking across the swarm. ## Configuration ### Rig-Level Config ```json // /config.json { "merge_queue": { "enabled": true, "target_branch": "main", "integration_branches": true, "on_conflict": "assign_back", "run_tests": true, "test_command": "go test ./...", "delete_merged_branches": true, "retry_flaky_tests": 1, "poll_interval": "30s", "max_concurrent": 1 } } ``` ### Per-Epic Overrides ```bash # Create epic with custom merge config bd create --type=epic --title="Risky refactor" \ --body="merge_config: run_tests: true test_command: 'go test -race ./...' on_conflict: assign_back" ``` ## Direct Landing (Bypass Queue) For single-polecat work or emergencies, Mayor can bypass the queue: ```bash gt land --direct / # This: # 1. Verifies polecat session is terminated # 2. Checks git state is clean # 3. Merges directly to main (no MR created) # 4. Closes the source issue # 5. Cleans up the branch ``` Direct landing skips the queue but still records the work in beads. ## Failure Recovery ### Engineer Crash If Engineer crashes mid-merge: 1. MR stays `in_progress` 2. On restart, Engineer queries `bd list --type=merge-request --status=in_progress` 3. For each, check git state and either complete or reset ### Partial Merge If merge succeeds but push fails: 1. Engineer retries push with exponential backoff 2. If persistent failure, roll back local merge 3. Mark MR as failed with reason ### Conflicting Merges If two MRs conflict: 1. First one wins (lands first) 2. Second gets conflict status 3. Worker rebases and resubmits 4. Dependencies prevent this for related work ## Observability ### Metrics - `mq_pending_count` - MRs waiting - `mq_processing_time` - Time from submit to merge - `mq_success_rate` - Merges vs rejections - `mq_conflict_rate` - How often conflicts occur - `mq_test_failure_rate` - Test failures ### Logs ``` [MQ] Processing gt-mr-abc123 (priority=P0, age=5m) [MQ] Fetching branch polecat/Nux/gt-xyz [MQ] No conflicts detected [MQ] Merging to main [MQ] Running tests: go test ./... [MQ] Tests passed (32s) [MQ] Pushing to origin [MQ] Merged: abc123def [MQ] Closed gt-mr-abc123 (reason=merged) [MQ] Closed source issue gt-xyz ``` ### Dashboard ``` ┌─────────────────────────────────────────────────────────────┐ │ MERGE QUEUE STATUS │ ├─────────────────────────────────────────────────────────────┤ │ Pending: 3 In Progress: 1 Merged (24h): 12 Failed: 2│ ├─────────────────────────────────────────────────────────────┤ │ QUEUE: │ │ ► gt-mr-004 P0 polecat/Nux/gt-xyz Processing... │ │ gt-mr-005 P1 polecat/Toast/gt-abc Ready │ │ gt-mr-006 P1 polecat/Capable/gt-def Blocked (005) │ │ gt-mr-007 P2 polecat/Nux/gt-ghi Ready │ ├─────────────────────────────────────────────────────────────┤ │ RECENT: │ │ ✓ gt-mr-003 Merged 5m ago (12s processing) │ │ ✓ gt-mr-002 Merged 18m ago (45s processing) │ │ ✗ gt-mr-001 Failed 22m ago (conflict with main) │ └─────────────────────────────────────────────────────────────┘ ``` ## Implementation Phases ### Phase 1: Schema & CLI (gt-kp2, gt-svi) - Define merge-request type in beads (or convention) - Implement `gt mq submit`, `gt mq list`, `gt mq status` - Manual Engineer processing (no automation yet) ### Phase 2: Engineer Loop (gt-3x1) - Implement processing loop - Conflict detection and handling - Test execution - Success/failure handling ### Phase 3: Integration Branches - `gt mq integration create/land/status` - Auto-targeting based on epic - Batch landing ### Phase 4: Advanced Features - Auto-rebase on conflict - Semantic merge (AI-assisted) - Parallel test execution - Cross-rig merge coordination ## Open Questions 1. **Beads schema extension**: Should merge-request be a first-class beads type, or just a convention (type field in description)? 2. **High-traffic rigs**: For very active rigs, should MRs go to a separate beads repo to reduce sync contention? 3. **Cross-rig merges**: If work spans multiple rigs, how do we coordinate? Federation design needed. 4. **Rollback**: If a merge causes problems, how do we track and revert? Need `gt mq revert` command.