Code Health & Technical Debt Cleanup

Comprehensive codebase cleanup to remove dead code, refactor monolithic files, deduplicate utilities, and improve maintainability. Based on ultrathink code health analysis conducted 2025-10-27.

Goals:
- Remove ~1,500 LOC of dead/unreachable code
- Split 2 monolithic files (server.go 2,273 LOC, sqlite.go 2,136 LOC) into focused modules
- Deduplicate scattered utility functions (normalizeLabels, BD_DEBUG checks)
- Consolidate test coverage (2,019 LOC of collision tests)
- Improve code navigation and reduce merge conflicts

Impact: Reduces codebase by ~6-8%, improves maintainability, faster CI/CD

Estimated Effort: 11 days across 4 phases


Update MCP server for hash IDs

Update beads-mcp server to support hash IDs and aliases.

## Changes Needed

### 1. MCP Function Signatures (No Change)
Functions already use issue IDs as strings, so they work with hash IDs:

```python
# These already work!
beads_show(issue_id: str)           # Accepts bd-af78e9a2 or #42
beads_update(issue_id: str, ...)    # Accepts both formats
beads_close(issue_ids: List[str])   # Accepts both formats
```

### 2. Add Alias Resolution Helper
File: integrations/beads-mcp/src/beads_mcp/server.py

```python
def resolve_issue_id(issue_id: str) -> str:
    """Resolve alias to hash ID if needed."""
    # Hash ID: pass through
    if issue_id.startswith('bd-') and len(issue_id) == 11:
        return issue_id

    # Alias: #42 or 42
    alias_str = issue_id.lstrip('#')
    try:
        alias = int(alias_str)
        # Call bd to resolve
        result = subprocess.run(
            ['bd', 'alias', 'find', f'bd-{alias}'],
            capture_output=True, text=True
        )
        if result.returncode == 0:
            return result.stdout.strip()
    except ValueError:
        pass

    # Invalid format
    raise ValueError(f"Invalid issue ID: {issue_id}")
```

### 3. Update Response Formatting
Show aliases in responses:

```python
@server.call_tool()
async def beads_show(issue_id: str) -> List[TextContent]:
    resolved_id = resolve_issue_id(issue_id)

    result = subprocess.run(['bd', 'show', resolved_id], ...)

    # Parse response and add alias info
    # Format: "bd-af78e9a2 (alias: #42)"
    ...
```

### 4. Add beads_alias_* Functions

```python
@server.call_tool()
async def beads_alias_list() -> List[TextContent]:
    """List all alias mappings."""
    result = subprocess.run(['bd', 'alias', 'list'], ...)
    return [TextContent(type="text", text=result.stdout)]

@server.call_tool()
async def beads_alias_set(alias: int, issue_id: str) -> List[TextContent]:
    """Manually assign alias to issue."""
    result = subprocess.run(['bd', 'alias', 'set', str(alias), issue_id], ...)
    return [TextContent(type="text", text=result.stdout)]

@server.call_tool()
async def beads_alias_compact() -> List[TextContent]:
    """Compact aliases to fill gaps."""
    result = subprocess.run(['bd', 'alias', 'compact'], ...)
    return [TextContent(type="text", text=result.stdout)]
```

### 5. Update Documentation
File: integrations/beads-mcp/README.md

```markdown
## Issue IDs (v2.0+)

The MCP server accepts both hash IDs and aliases:

```python
# Using hash IDs
await beads_show(issue_id="bd-af78e9a2")

# Using aliases
await beads_show(issue_id="#42")
await beads_show(issue_id="42")  # Shorthand
```

## Alias Management

New functions for alias control:

- `beads_alias_list()` - List all alias mappings
- `beads_alias_set(alias, issue_id)` - Manually assign alias
- `beads_alias_compact()` - Compact aliases to fill gaps

## Migration

After migrating to hash IDs:
1. Update beads-mcp: `pip install --upgrade beads-mcp`
2. Restart MCP server
3. All existing workflows continue to work
```

### 6. Version Compatibility
Detect and handle both v1.x and v2.0 formats:

```python
def detect_beads_version() -> str:
    """Detect if beads is using sequential or hash IDs."""
    result = subprocess.run(['bd', 'list', '-n', '1'], ...)
    first_id = parse_first_issue_id(result.stdout)

    if first_id.startswith('bd-') and len(first_id) > 11:
        return '2.0'  # Hash ID
    else:
        return '1.x'  # Sequential ID

# On startup
beads_version = detect_beads_version()
logger.info(f"Detected beads version: {beads_version}")
```

## Testing

### Unit Tests
File: integrations/beads-mcp/tests/test_hash_ids.py

```python
def test_resolve_hash_id():
    """Hash IDs pass through unchanged."""
    assert resolve_issue_id("bd-af78e9a2") == "bd-af78e9a2"

def test_resolve_alias():
    """Aliases resolve to hash IDs."""
    # Mock bd alias find command
    assert resolve_issue_id("#42") == "bd-af78e9a2"
    assert resolve_issue_id("42") == "bd-af78e9a2"

def test_invalid_id():
    """Invalid IDs raise ValueError."""
    with pytest.raises(ValueError):
        resolve_issue_id("invalid")
```

### Integration Tests
```python
async def test_show_with_hash_id(mcp_server):
    result = await mcp_server.beads_show(issue_id="bd-af78e9a2")
    assert "bd-af78e9a2" in result[0].text

async def test_show_with_alias(mcp_server):
    result = await mcp_server.beads_show(issue_id="#42")
    assert "bd-af78e9a2" in result[0].text  # Resolved
```

## Backward Compatibility
The MCP server should work with both:
- Beads v1.x (sequential IDs)
- Beads v2.0+ (hash IDs)

Detection happens at runtime based on issue ID format.

## Files to Modify
- integrations/beads-mcp/src/beads_mcp/server.py
- integrations/beads-mcp/README.md
- integrations/beads-mcp/tests/test_hash_ids.py (new)
- integrations/beads-mcp/pyproject.toml (bump version)

## Deployment
```bash
cd integrations/beads-mcp
# Bump version to 2.0.0
poetry version 2.0.0
# Publish to PyPI
poetry publish --build
```


Add t.Parallel() to slow hash multiclone tests

Add t.Parallel() to TestHashIDs_MultiCloneConverge and TestHashIDs_IdenticalContentDedup so they run concurrently.

Expected savings: ~10 seconds (from 20s to ~11s)

Implementation:
- Add t.Parallel() call at start of each test function
- Verify tests don't share resources that would cause conflicts
- Run tests to confirm they work in parallel

File: beads_hash_multiclone_test.go:34, :101


Delete collision resolution code

Remove ~2,100 LOC of ID collision detection and resolution code (no longer needed with hash IDs).

## Files to Delete Entirely
```
internal/storage/sqlite/collision.go              (~800 LOC)
internal/storage/sqlite/collision_test.go         (~300 LOC)
cmd/bd/autoimport_collision_test.go               (~400 LOC)
```

## Code to Remove from Existing Files

### internal/importer/importer.go
Remove:
- `DetectCollisions()` calls
- `ScoreCollisions()` logic
- `RemapCollisions()` calls
- `handleRename()` function
- All collision-related error handling

Keep:
- Basic import logic
- Exact match detection (idempotent import)

### beads_twoclone_test.go
Remove:
- `TestTwoCloneCollision` (bd-71107098)
- `TestThreeCloneCollision` (bd-cbed9619)
- `TestFiveCloneCollision` (bd-a40f374f)
- All N-way collision tests

### cmd/bd/import.go
Remove:
- `--resolve-collisions` flag
- `--dry-run` collision preview
- Collision reporting

## Issues Closed by This Change
- bd-71107098: Add test for symmetric collision
--89: Content-hash collision resolution
- bd-cbed9619: N-way collision resolution epic
- bd-cbed9619.5: Add ScoreCollisions (already done but now unnecessary)
- bd-cbed9619.4: Make DetectCollisions read-only
- bd-cbed9619.3: ResolveNWayCollisions function
- bd-cbed9619.2: Multi-round import convergence
- bd-cbed9619.1: Multi-round convergence for N-way collisions
- bd-e6d71828: Transaction + retry logic for collisions
- bd-70419816: Test case for symmetric collision

## Verification Steps
1. `grep -r "collision" --include="*.go"` → should only find alias conflicts
2. `go test ./...` → all tests pass
3. `go build ./cmd/bd` → clean build
4. Check LOC reduction: `git diff --stat`

## Expected Metrics
- **Files deleted**: 3
- **LOC removed**: ~2,100
- **Test coverage**: Should increase (less untested code)
- **Binary size**: Slightly smaller

## Caution
Do NOT delete:
- Alias conflict resolution (new code in bd-f8b764c9.6)
- Duplicate detection (bd-581b80b3, bd-149) - different from ID collisions
- Merge conflict resolution (bd-7e7ddffa.1, bd-5f483051) - git conflicts, not ID collisions

## Files to Modify
- internal/importer/importer.go (remove collision handling)
- cmd/bd/import.go (remove --resolve-collisions flag)
- beads_twoclone_test.go (remove collision tests)
- Delete: internal/storage/sqlite/collision.go
- Delete: internal/storage/sqlite/collision_test.go
- Delete: cmd/bd/autoimport_collision_test.go

## Testing
- Ensure all remaining tests pass
- Manual test: create issue on two clones, sync → no collisions
- Verify error if somehow hash collision occurs (extremely unlikely)


GetNextChildID doesn't attempt parent resurrection from JSONL history

When creating a child issue with --parent flag, GetNextChildID fails immediately if parent doesn't exist in DB, without attempting to resurrect it from JSONL history. This breaks the intended resurrection workflow and causes 'parent issue X does not exist' errors even when the parent exists in JSONL.

Related to GH #334 and #278.

Current behavior:
- GetNextChildID checks if parent exists in DB
- If not found, returns error immediately
- No resurrection attempt

Expected behavior:
- GetNextChildID should call TryResurrectParent before failing
- Parent should be restored as tombstone if found in JSONL history
- Child creation should succeed if resurrection succeeds

Impact: Users cannot create child issues for parents that were deleted but exist in JSONL history.


Document which files to commit after bd init --branch

GH #312 reported confusion about which files should be committed after running 'bd init --branch beads-metadata'. Updated PROTECTED_BRANCHES.md to clearly document:

1. Files that should be committed to protected branch (main):
   - .beads/.gitignore
   - .gitattributes

2. Files that are automatically gitignored
3. Files that live in the sync branch (beads-metadata)

Changes:
- Added step-by-step instructions in Quick Start section
- Added 'What lives in each branch' section to How It Works
- Clarified the directory structure diagram

Fixes: https://github.com/steveyegge/beads/issues/312


Audit Current Cache Usage

Understand exactly what code depends on the storage cache

Audit complete. See CACHE_AUDIT.md for full findings.

Summary:
- Cache was already removed in commit 322ab63 (2025-10-28)
- server_cache_storage.go deleted (~286 lines)
- All getStorageForRequest calls replaced with s.storage
- All environment variables removed
- MCP multi-repo works via per-project daemon architecture
- All tests updated and passing
- Only stale comment in server.go needed fixing


Performance Testing Documentation

Create docs/performance-testing.md documenting the performance testing framework.

Sections:
1. Overview - What the framework does, goals
2. Running Benchmarks
   - make bench command
   - Running specific benchmarks
   - Interpreting output (ns/op, allocs/op)
3. Profiling and Analysis
   - Viewing CPU profiles with pprof
   - Reading flamegraphs
   - Memory profiling
   - Finding hotspots
4. User Diagnostics
   - bd doctor --perf usage
   - Sharing profiles with bug reports
   - Understanding the report output
5. Comparing Performance
   - Using benchstat for before/after comparisons
   - Detecting regressions
6. Tips for Optimization
   - Common patterns
   - When to profile vs benchmark

Style:
- Concise, practical examples
- Screenshots/examples of pprof output
- Clear command-line examples
- Focus on workflow, not theory


Extract computeJSONLHash helper to eliminate code duplication

SHA256 hash computation is duplicated in 3 places:
- cmd/bd/integrity.go:50-52
- cmd/bd/sync.go:611-613
- cmd/bd/import.go:318-319

Extract to shared helper function computeJSONLHash(jsonlPath string) (string, error) that includes proper #nosec comment and error handling.


Test database naming


BG's issue to reopen


bd-hv01: Missing cleanup of .merged temp file on failure

Problem: deletion_tracking.go:49 creates tmpMerged file but does not clean up on failure, causing disk space leak and potential interference with subsequent syncs.

Fix: Add defer os.Remove(tmpMerged) after creating temp file path.

Files: cmd/bd/deletion_tracking.go:38-89


Migration tool: sequential → hash IDs

Create migration tool to convert existing sequential-ID databases to hash-ID format.

## Command: bd migrate --hash-ids

```bash
bd migrate --hash-ids [--dry-run]
```

## Migration Process

### 1. Backup Database
```bash
cp .beads/beads.db .beads/beads.db.backup-1761798384
echo "✓ Backup created: .beads/beads.db.backup-1234567890"
```

### 2. Generate Hash IDs for Existing Issues
```go
func migrateToHashIDs(db *SQLiteStorage) error {
    // Read all issues
    issues, err := db.ListIssues(ctx, ListOptions{Status: "all"})

    // Generate mapping: old ID → new hash ID
    mapping := make(map[string]string)
    for _, issue := range issues {
        hashID := GenerateHashID(
            issue.Title,
            issue.Description,
            issue.CreatedAt,
            db.workspaceID,
        )
        mapping[issue.ID] = hashID
    }

    // Detect hash collisions (extremely rare)
    if hasCollisions(mapping) {
        return fmt.Errorf("hash collision detected, aborting")
    }

    return nil
}
```

### 3. Update Database Schema
```sql
-- Add alias column
ALTER TABLE issues ADD COLUMN alias INTEGER UNIQUE;

-- Populate aliases from old IDs
UPDATE issues SET alias = CAST(SUBSTR(id, 4) AS INTEGER)
  WHERE id LIKE 'bd-%' AND SUBSTR(id, 4) GLOB '[0-9]*';

-- Create new issues_new table with hash IDs
CREATE TABLE issues_new (
    id TEXT PRIMARY KEY,  -- Hash IDs now
    alias INTEGER UNIQUE,
    title TEXT NOT NULL,
    -- ... rest of schema
);

-- Copy data with ID mapping
INSERT INTO issues_new SELECT
    <new_hash_id>,  -- From mapping
    alias,
    title,
    -- ...
FROM issues;

-- Drop old table, rename new
DROP TABLE issues;
ALTER TABLE issues_new RENAME TO issues;
```

### 4. Update Dependencies
```sql
-- Update depends_on_id using mapping
UPDATE dependencies
SET issue_id = <new_hash_id>,
    depends_on_id = <new_depends_on_hash_id>
FROM mapping;
```

### 5. Update Text References
```go
// Update all text fields that mention old IDs
func updateTextReferences(db *SQLiteStorage, mapping map[string]string) error {
    for oldID, newID := range mapping {
        // Update description, notes, design, acceptance_criteria
        db.Exec(`UPDATE issues SET
            description = REPLACE(description, ?, ?),
            notes = REPLACE(notes, ?, ?),
            design = REPLACE(design, ?, ?),
            acceptance_criteria = REPLACE(acceptance_criteria, ?, ?)
        `, oldID, newID, oldID, newID, oldID, newID, oldID, newID)
    }
}
```

### 6. Export to JSONL
```bash
bd export  # Writes hash IDs to .beads/issues.jsonl
git add .beads/issues.jsonl
git commit -m "Migrate to hash-based IDs"
```

## Output
```bash
$ bd migrate --hash-ids
Migrating to hash-based IDs...
✓ Backup created: .beads/beads.db.backup-1730246400
✓ Generated 150 hash IDs
✓ No hash collisions detected
✓ Updated issues table schema
✓ Updated 150 issue IDs
✓ Updated 87 dependencies
✓ Updated 234 text references
✓ Exported to .beads/issues.jsonl
✓ Migration complete!

Next steps:
  1. Test: bd list, bd show #1, etc.
  2. Commit: git commit -m "Migrate to hash-based IDs"
  3. Push: git push origin main
  4. Notify collaborators to pull and re-init
```

## Dry Run Mode
```bash
$ bd migrate --hash-ids --dry-run
[DRY RUN] Would migrate 150 issues:
  bd-1c63eb84   → bd-af78e9a2  (alias: #1)
  bd-9063acda   → bd-e5f6a7b8  (alias: #2)
  ...
  bd-150 → bd-9a8b7c6d  (alias: #150)

[DRY RUN] Would update:
  - 150 issue IDs
  - 87 dependencies
  - 234 text references in descriptions/notes

No changes made. Run without --dry-run to apply.
```

## Files to Create
- cmd/bd/migrate.go (new command)
- internal/storage/sqlite/migrations/hash_ids.go

## Testing
- Test migration on small database (10 issues)
- Test migration on large database (1000 issues)
- Test hash collision detection (inject collision artificially)
- Test text reference updates
- Test rollback (restore from backup)
- Test migrated database works correctly

## Rollback Procedure
```bash
# If migration fails or has issues
mv .beads/beads.db.backup-1234567890 .beads/beads.db
bd export  # Restore JSONL from backup DB
```

## Multi-Clone Coordination
**Important**: All clones must migrate before syncing:

1. Coordinator sends message: "Migrating to hash IDs on 2025-10-30 at 10:00 UTC"
2. All collaborators pull latest changes
3. All run: `bd migrate --hash-ids`
4. All push changes
5. New work can continue with hash IDs

**Do NOT**:
- Mix sequential and hash IDs in same database
- Sync before all clones migrate


Migration and onboarding for multi-repo

Create migration tools, wizards, and documentation to help users adopt multi-repo workflow, with special focus on OSS contributor onboarding and team adoption scenarios.


bd cleanup fails with CHECK constraint on status/closed_at mismatch

## Problem

Running `bd cleanup --force` fails with:

```
Error: failed to create tombstone for bd-okh: sqlite3: constraint failed: CHECK constraint failed: (status = 'closed') = (closed_at IS NOT NULL)
```

## Root Cause

The database has a CHECK constraint ensuring closed issues have `closed_at` set and non-closed issues don't. When `bd cleanup` tries to convert a closed issue to a tombstone, this constraint fails if:
1. The issue has status='closed' but closed_at is NULL, OR
2. The issue is being converted but the tombstone creation doesn't properly handle the closed_at field

## Impact

- `bd cleanup --force` cannot complete
- 722 closed issues cannot be cleaned up
- Blocks routine maintenance

## Investigation Needed

1. Check the bd-okh record in the database:
   ```sql
   SELECT id, status, closed_at, deleted_at FROM issues WHERE id = 'bd-okh';
   ```
2. Determine if this is data corruption or a bug in tombstone creation

## Proposed Solutions

1. **If data corruption**: Add a `bd doctor --fix` check that repairs status/closed_at mismatches
2. **If code bug**: Fix the tombstone creation to properly handle the constraint

## Files to Investigate

- `internal/storage/sqlite/sqlite.go` - DeleteIssue / tombstone creation
- Schema CHECK constraint definition

## Acceptance Criteria

- [ ] Identify root cause (data vs code bug)
- [ ] `bd cleanup --force` completes successfully
- [ ] `bd doctor` detects status/closed_at mismatches


Event-driven daemon architecture

Replace 5-second polling sync loop with event-driven architecture that reacts instantly to changes. Eliminates stale data issues while reducing CPU ~60%. Key components: FileWatcher (fsnotify), Debouncer (500ms), RPC mutation events, optional git hooks. Target latency: <500ms (vs 5000ms). See event_driven_daemon.md for full design.

Production-ready after 3 critical fixes (commit 349b892):
- Skip redundant imports (mtime check prevents self-trigger loops)
- Add server.Stop() in serverErrChan case (clean shutdown)
- Fallback ticker (60s) when watcher unavailable (ensures remote sync)

Ready to make default after integration test (bd-1f4086c5.1) passes.


Implement signal-aware context in CLI commands

Replace context.Background() with signal.NotifyContext() to enable graceful cancellation.

## Context
Part of context propagation work (bd-350). Phase 1 infrastructure is complete - sqlite.New() now accepts context parameter.

## Implementation
Set up signal-aware context at the CLI entry points:

```go
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel()
```

## Key Locations
- cmd/bd/main.go:438 (marked with TODO(bd-350))
- cmd/bd/daemon.go:317 (marked with TODO(bd-350))
- Other command entry points

## Benefits
- Ctrl+C cancels ongoing database operations
- Graceful shutdown on SIGTERM
- Better user experience during long operations

## Acceptance Criteria
- [ ] Ctrl+C during import cancels operation cleanly
- [ ] Ctrl+C during export cancels operation cleanly
- [ ] No database corruption on cancellation
- [ ] Proper cleanup on signal (defers execute)


Test parent issue


Document deletions manifest in AGENTS.md and README

Parent: bd-imj

## Task
Add documentation about the deletions manifest feature.

## Locations to Update

### AGENTS.md
- Explain that deletions.jsonl is tracked in git
- Document that `bd delete` records to the manifest
- Explain cross-clone propagation mechanism

### README.md
- Brief mention in .beads directory structure section
- Link to detailed docs if needed

### docs/deletions.md (new file)
- Full technical documentation
- Format specification
- Pruning policy
- Git history fallback
- Troubleshooting

## Acceptance Criteria
- AGENTS.md updated with deletion workflow
- README.md mentions deletions.jsonl purpose
- New docs/deletions.md with complete reference


Implement "bd daemons logs" subcommand

Add command to view daemon logs for a specific workspace. Requires daemon logging to file (may need separate issue for log infrastructure).


Update AGENTS.md with event-driven mode

Document BEADS_DAEMON_MODE env var. Explain opt-in during Phase 1. Add troubleshooting for watcher failures.


Investigate database pollution - unexpected issue count increases

Two repositories showing unexpected issue counts:
- ~/src/beads: 280 issues (expected ~209-220)
- ~/src/dave/beads: 895 issues (675 open, 149 closed)

This suggests database pollution - issues from one repository leaking into another. Need to investigate:
1. Run bd detect-pollution on both repos
2. Check for cross-repo contamination
3. Identify source of pollution (daemon? multi-repo config? import issues?)
4. Clean up polluted databases
5. Prevent future pollution


Fix FOREIGN KEY constraint failures in AddComment and ApplyCompaction

The 'CloseIssue', 'UpdateIssueID', and 'RemoveLabel' methods were fixed in PR #348 to prevent foreign key constraint failures when operating on non-existent issues.

However, the Oracle identified two other methods that follow the same problematic pattern:
1. `AddComment` (in `internal/storage/sqlite/events.go`)
2. `ApplyCompaction` (in `internal/storage/sqlite/compact.go`)

These methods attempt to insert an event record after updating the issue, without verifying that the issue update actually affected any rows. This leads to a foreign key constraint failure if the issue does not exist.

We need to:
1. Create reproduction tests for these failure cases
2. Apply the same fix pattern: check `RowsAffected()` after the update, and return a proper "issue not found" error if it is 0, before attempting to insert the event.
3. Standardize the error message format to "issue %s not found" or "issue not found: %s" for consistency.


Investigate TestRoutingIntegration failure

TestRoutingIntegration/maintainer_with_SSH_remote failed during pre-commit check with "expected role maintainer, got contributor".
This occurred while running `go test -short ./...` on darwin/arm64.
The failure appears unrelated to storage/sqlite changes.
Need to investigate if this is a flaky test or environmental issue.


Add .beads/README.md during 'bd init' for project documentation and promotion

When 'bd init' is run, automatically generate a .beads/README.md file that:

1. Briefly explains what Beads is (AI-native issue tracking that lives in your repo)
2. Links to the main repository: https://github.com/steveyegge/beads
3. Provides a quick reference of essential commands:
   - bd create: Create new issues
   - bd list: View all issues
   - bd update: Modify issue status/details
   - bd show: View issue details
   - bd sync: Sync with git remote
4. Highlights key benefits for AI coding agents and developers
5. Encourages developers to try it out

The README should be enthusiastic and compelling to get open source contributors excited about using Beads for their AI-assisted development workflows.