Make auto-flush debounce duration configurable

flushDebounce is hardcoded to 5 seconds. Make it configurable via environment variable BEADS_FLUSH_DEBOUNCE (e.g., '500ms', '10s'). Current 5-second value is reasonable for interactive use, but CI/automated scenarios might want faster flush. Add getDebounceDuration() helper function. Located in cmd/bd/main.go:31.


Make beads reusable as a Go library for external projects like vc

Currently beads is only usable as a CLI tool. We want to use beads as a library in other Go projects like ~/src/vc so they can programmatically manage issues without shelling out to the bd CLI.

Goals:
- Export public API from internal packages
- Document Go package usage
- Provide examples of programmatic usage
- Ensure vc can import and use beads storage layer directly

Use case: The vc project needs issue tracking and wants to use beads as an embedded library rather than as a separate CLI tool.

UnderlyingDB() method implemented and tested. Core functionality complete. Still needs documentation updates (bd-253) and lifecycle safety enhancements (bd-223).


GH#146: No color showing in terminal for some users

User reports color not working in macOS (Taho 26.0.1) with iTerm 3.6.4 and Terminal.app, despite color working elsewhere in terminal. Python rich and printf escape codes work.

Need to investigate:
- Is NO_COLOR env var set?
- Terminal type detection?
- fatih/color library configuration
- Does bd list show colors? bd ready? bd init?
- What's the output of: echo $TERM, echo $NO_COLOR


Fix nil pointer crash in bd reopen command

bd reopen crashes with SIGSEGV at reopen.go:30. Nil pointer dereference when trying to reopen an issue.

Fixed by adding daemon RPC support to reopen command. Pattern: check daemonClient != nil first, use RPC UpdateArgs with Status=open, fall back to direct store if daemon unavailable.


Address gosec security warnings (102 issues)

Security linter warnings: file permissions (0755 should be 0750), G304 file inclusion via variable, G204 subprocess launches. Many are false positives but should be reviewed.

Review each gosec warning. Add exclusions for legitimate cases to .golangci.yml. Fix real security issues (overly permissive file modes).


Multi-project MCP context switching (GH#145)

Enable MCP server to manage multiple beads projects in a single session with per-request workspace_root parameter.

Current bug: set_context(workspace_root) doesn't actually switch sockets - all operations hit first initialized socket.

Use case: Managing tasks across multiple organizations with different permission models (internal, partners, open source).

Architecture: Connection pool keyed by workspace_root, each maintaining its own daemon socket connection. Request-scoped routing using ContextVar to avoid global state races.

See GH#145 for full requirements and user context.

✅ APPROVED WITH MODIFICATIONS by architectural review (2025-10-25)

LSP-style model is correct: Single MCP server → per-project daemons → isolated databases.

CRITICAL CHANGES from review:
1. **Simplify connection pool**: No LRU eviction initially (typical user: 2-5 projects)
2. **Add asyncio.Lock**: Prevent race conditions in pool access
3. **Defer health checks**: Only retry on failure, not preemptive pings
4. **Handle submodules**: Check local .beads BEFORE git toplevel
5. **Path canonicalization**: realpath + git toplevel with caching

RISKS MITIGATED:
- Global _client bug: Replace with connection pool keyed by canonical path
- Race conditions: Add asyncio.Lock for pool mutations
- Submodule edge case: Check .beads directory first
- Stale sockets: Retry once on connection failure

Estimated effort: 2.5-3.5 days (simplified from 2.5-4.5 days)
Confidence: 8/10

- Multiple projects can be accessed in single MCP session
- Per-request workspace_root parameter works on all tools
- No cross-project data leakage
- Concurrent calls to different projects work correctly
- Stale sockets auto-reconnect with retry/backoff
- Integration tests verify isolation across 2+ temp repos
- set_context() still works as default fallback

Review doc: docs/bd-265-architectural-review.md

Implementation order:
1. bd-255 (connection manager) - Foundation with pool + lock
2. bd-237 (ContextVar routing) - Per-request workspace
3. bd-240 (require_context) - Validation
4. bd-238 (tests) - Concurrency + edge cases
5. bd-256 (docs) - Usage guide
6. [deleted:bd-142] (health checks) - DEFERRED to Phase 2


Implement request-scoped routing with ContextVar

Add ContextVar-based routing to avoid global state races during concurrent multi-project calls.

Approach:
- Define current_workspace: ContextVar[str|None] in server.py
- Add @with_workspace decorator that resolves workspace_root (via _resolve_workspace_root + realpath)
- Set ContextVar for duration of tool call, reset after
- Falls back to set_context default (BEADS_WORKING_DIR) if workspace_root not provided
- beads_mcp.tools.get_client() reads current_workspace from ContextVar

Blocks: bd-265 (connection manager must exist first)

Decorator pattern with ContextVar for request-scoped workspace routing.

@with_workspace decorator:
- Extract workspace_root parameter from tool call
- Resolve via _resolve_workspace_root + realpath
- Set current_workspace ContextVar for request duration
- Falls back to BEADS_WORKING_DIR if workspace_root not provided
- Reset ContextVar after tool completes

Applied to all tools in server.py. _get_client() reads current_workspace.

⚠ CONCURRENCY GOTCHA (from architectural review):
- ContextVar doesn't propagate to asyncio.create_task() spawned tasks
- SOLUTION: Keep tool calls synchronous, no background task spawning
- If background tasks needed: use contextvars.copy_context()

Document this limitation in bd-256.

Blocks on bd-255 (connection pool must exist first).

CRITICAL: Do NOT spawn background tasks within tool implementations.
ContextVar propagation to spawned tasks is unreliable.


Add integration tests for multi-project MCP switching

Comprehensive tests to verify multi-project isolation, concurrency, and edge cases.

EXPANDED TEST COVERAGE (per architectural review):

**Concurrency tests (CRITICAL):**
- asyncio.gather() with calls to different workspace_root values
- Verify no cross-project data leakage
- Verify pool lock prevents race conditions

**Edge case tests:**
- Submodule handling: Parent repo vs submodule with own .beads
- Symlink deduplication: Same physical path via different symlinks
- Stale socket recovery: Kill daemon, verify retry on failure
- Missing .beads directory handling

**Isolation tests:**
- Create 2+ temp repos with bd init
- Verify operations in project A don't affect project B
- Stress test: many parallel calls across 3-5 repos

Estimated effort: M (1-2 days) including fixtures for temp repos and daemon process management

Test structure:

1. test_concurrent_multi_project.py:
   - asyncio.gather with 2+ projects
   - Verify pool lock prevents corruption

2. test_path_canonicalization.py:
   - Submodule edge case (check .beads first)
   - Symlink deduplication (realpath normalization)

3. test_stale_socket_recovery.py:
   - Kill daemon mid-session
   - Verify retry-on-failure works

4. test_cross_project_isolation.py:
   - Create issues in project A
   - List from project B, verify empty
   - No data leakage

Use pytest fixtures for temp repos and daemon lifecycle.

- All concurrency tests pass with asyncio.gather
- Submodule edge case handled correctly
- Symlinks deduplicated to same connection
- Stale socket retry works
- No cross-project data leakage in stress tests


Fix require_context to support per-request workspace_root

Update require_context decorator to pass if either:
- workspace_root was provided on the tool call (via ContextVar), OR
- BEADS_WORKING_DIR is set (from set_context)

Stop using BEADS_DB as router - treat it only as CLI fallback.

Ensures backward compatibility with set_context() while supporting new per-request routing.


Add health checks and reconnection logic for stale sockets

Handle stale/broken socket connections after daemon restarts or upgrades.

Features:
- ping/health_check method on client
- Check before use or periodic health check
- On failure: drop from pool, attempt reconnect with exponential backoff
- Clear error propagation if reconnect fails after retries
- Handle version mismatch after daemon upgrade
- Handle long-idle connections closed by daemon

Add async ping() to DaemonClient. ConnectionManager.get_client() pings before returning cached client. On failure, del from pool and retry connect. Bounded retry (3-5 attempts) with backoff.


Add connection manager for per-project daemon sockets

Create connection pool for managing multiple daemon socket connections in beads_mcp.tools.

SIMPLIFIED APPROACH (per architectural review):
- Connection pool dict[str, BdClientBase] keyed by canonical path
- Add asyncio.Lock for thread-safe pool access (CRITICAL)
- Path canonicalization with @lru_cache for performance
- NO LRU eviction (start simple, add if monitoring shows memory issues)
- NO preemptive health checks (only retry on failure)

Features:
- get_client(workspace_root) → returns client for that project
- Canonicalization: realpath + git toplevel (handles symlinks)
- Submodule-aware: Check local .beads BEFORE git toplevel
- Auto-start daemon if socket missing (existing behavior)
- Validate version on connect (existing behavior)

CRITICAL: Replace global _client singleton with pool to fix set_context() bug.

Implementation in tools.py:

```python
from contextvars import ContextVar
import asyncio
from functools import lru_cache

current_workspace: ContextVar[str | None] = ContextVar('workspace', default=None)
_connection_pool: Dict[str, BdClientBase] = {}
_pool_lock = asyncio.Lock()  # Prevent race conditions

@lru_cache(maxsize=128)
def _canonicalize_path(path: str) -> str:
    # 1. realpath to resolve symlinks
    real = os.path.realpath(path)

    # 2. Check local .beads (submodule edge case)
    if os.path.exists(os.path.join(real, ".beads")):
        return real

    # 3. Try git toplevel
    return _resolve_workspace_root(real)

async def _get_client() -> BdClientBase:
    workspace = current_workspace.get() or os.environ.get("BEADS_WORKING_DIR")
    if not workspace:
        raise BdError("No workspace set")

    canonical = _canonicalize_path(workspace)

    async with _pool_lock:  # CRITICAL: prevent concurrent mutations
        if canonical not in _connection_pool:
            _connection_pool[canonical] = create_bd_client(
                prefer_daemon=True,
                working_dir=canonical
            )
        return _connection_pool[canonical]
```

~200 LOC total (simplified from ~400-500)


Update MCP multi-project documentation

Update documentation for multi-project workflow in README.md, AGENTS.md, and MCP integration docs.

EXPANDED SECTIONS (per architectural review):

**Usage examples:**
- Per-request workspace_root parameter usage
- Concurrent multi-project queries with asyncio.gather
- Migration from set_context() to workspace_root parameter

**Architecture notes:**
- Connection pooling behavior (no limits initially)
- set_context() as default fallback (still supported)
- Library users NOT affected (all changes in MCP layer)

**Concurrency gotchas (CRITICAL):**
- ContextVar doesn't propagate to asyncio.create_task()
- Do NOT spawn background tasks in tool implementations
- All tool calls should be synchronous/sequential

**Troubleshooting:**
- Stale sockets (retry once on failure)
- Version mismatches (auto-detected since v0.16.0)
- Path aliasing via symlinks (deduplicated by realpath)
- Submodules with own .beads (handled correctly)

**Use cases:**
- Multi-organization collaboration (GH#145)
- Parallel project management scripts
- Cross-project queries

Documentation structure:

1. integrations/beads-mcp/README.md:
   - Add "Multi-Project Support" section
   - workspace_root parameter examples
   - Connection pool behavior

2. AGENTS.md:
   - Update MCP section with workspace_root usage
   - Add concurrency warning (no spawned tasks)
   - Document library non-impact

3. New: docs/MCP_MULTI_PROJECT.md:
   - Detailed architecture explanation
   - Migration guide from set_context()
   - Troubleshooting guide
   - Edge cases (submodules, symlinks)


Beads Library Integration

Migrate from custom SQLite implementation to using Beads as a library dependency. This eliminates ~3000 lines of duplicated code, reduces schema drift risk, and automatically inherits new Beads features.

**Key Benefits:**
- Remove 3000+ lines of duplicated SQLite code
- Eliminate schema drift between bd and vc CLIs
- Inherit Beads improvements automatically
- Stronger type safety with Beads error types
- Faster development velocity for new features
- Clean separation: Beads stays general-purpose, VC extends via wrapper

**Architecture Principle:**
Beads remains 100% standalone with NO VC dependencies. VC imports Beads (VC → Beads dependency) and wraps it with VC-specific storage methods. Both share the same database but maintain separate table namespaces.

**Current Pain Points:**
1. Code Duplication: Issue CRUD, dependency graphs, labels, status transitions all reimplemented
2. Schema Drift Risk: VC schema manually defined, could diverge from Beads
3. Lost Features: Can't leverage Beads query optimizer or advanced features without process spawning
4. Atomic Operations: Hand-rolled 100+ line transaction management
5. Maintenance Burden: Every Beads feature must be manually replicated

**Concrete Example:**
When Beads adds a new field (e.g., estimated_hours):
- Current: 4-6 hours of manual work (update types, 6+ SQL queries, migration, testing)
- With Library: 5 minutes (go get -u github.com/steveyegge/beads)

**Phased Approach:**
1. Phase 1: Add Beads dependency (non-breaking, feature flag)
2. Phase 2: Implement VCStorage wrapper (embeds beads.Storage)
3. Phase 3: Migration script for existing databases
4. Phase 4: Gradual cutover, deprecate SQLite code

**Related Analysis:**
See BEADS_INTEGRATION_ANALYSIS.md for detailed current state analysis and BEADS_LIBRARY_INTEGRATION_EPIC.md for full design document (both to be archived after issue creation).

**Estimated Effort:** 3-4 sprints
**Priority:** P2 (Medium-High - architectural improvement, high ROI)

Phase 1 (bd-267) complete! Beads can now be used as a Go library. VC can import github.com/steveyegge/beads and use beads.Storage directly instead of spawning CLI processes. No custom tables needed - VC uses pure Beads primitives.


bd daemon auto-sync can wipe out issues.jsonl when database is empty

During dogfooding session, bd daemon auto-sync exported empty database to JSONL, losing all 177 issues. Had to git restore to recover.

Root cause: bd export doesn't check if database is empty before exporting. When daemon has empty/wrong database, it wipes out valid JSONL file.

Impact: DATA LOSS

Add safeguard in bd export:
1. Count total issues in database before export
2. If count is 0, refuse to export and show error
3. Provide --force flag to override if truly want empty export

Alternative: Check if target JSONL exists and has issues, warn if about to replace with empty export

- bd export refuses to export when database has 0 issues
- Clear error message: "Refusing to export empty database (0 issues). Use --force to override."
- --force flag allows override for intentional empty exports
- Test: export with empty db fails, export with --force succeeds


bd import doesn't update database modification time (WAL mode)

When running bd import in WAL mode, the -wal file is updated but main .db file timestamp stays old. This breaks staleness detection which only checks main .db file.

Discovered during dogfooding when import didn't trigger staleness refresh.

Impact: Staleness checks fail to detect that database is newer than expected

Two options:
1. Checkpoint WAL after import to flush changes to main .db file
2. Update staleness detection to check both .db and -wal file timestamps

Option 1 is simpler and safer - just add PRAGMA wal_checkpoint(FULL) after import completes

- After bd import, main .db file modification time is updated
- Staleness detection correctly sees database as fresh
- Test: import, check .db mtime, verify it's recent


bd should show which database file it's using

During dogfooding, bd showed "0 issues" when correct database had 177 issues. Confusion arose from which database path was being used (daemon default vs explicit --db flag).

Users need clear feedback about which database file bd is actually using, especially when daemon is involved.

Impact: User confusion, working with wrong database unknowingly

Add database path to verbose output or as a bd info command:
1. bd info shows current database path, daemon status
2. OR: bd ready/list/etc --verbose shows "Using database: /path/to/.beads/beads.db"
3. Consider adding to bd status output

When database path differs from expected, show warning

- User can easily determine which database file bd is using
- bd info or similar command shows full database path
- When using unexpected database (e.g., daemon vs explicit --db), show clear indication
- Documentation updated with how to check database path


Implement configurable sort policy for GetReadyWork

Add SortPolicy field to WorkFilter to support different sorting strategies:
- hybrid (default): Recent issues by priority, old by age
- priority: Strict priority ordering for autonomous execution
- oldest: Backlog clearing mode

Solves issue where VC executor selects low-priority work instead of critical P0 work.

See SORT_POLICY_DESIGN.md for complete design spec including:
- Type definitions and constants
- SQL ORDER BY clause generation
- Testing strategy with test cases
- CLI integration with --sort flag
- Migration plan and backwards compatibility

- SortPolicy type added to types.go
- buildOrderByClause() implemented in ready.go
- Unit tests for all 3 policies pass
- Integration tests verify priority selection order
- bd ready --sort priority|oldest|hybrid works
- Empty SortPolicy defaults to hybrid (backwards compatible)
- Documentation updated in README.md and WORKFLOW.md


Add configurable SortPolicy to GetReadyWork

Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.

See SORT_POLICY_DESIGN.md for complete design.

Implementation summary:
1. Add SortPolicy type and constants (hybrid, priority, oldest)
2. Add SortPolicy field to WorkFilter
3. Implement buildOrderByClause() to generate SQL based on policy
4. Default to hybrid for backwards compatibility
5. Add --sort flag to bd ready command

This enables autonomous execution systems (like VC) to use strict priority ordering while preserving the current hybrid behavior for interactive use.

Unit tests verify each policy generates correct ORDER BY. Integration tests verify priority, hybrid, and oldest policies select issues in expected order. CLI bd ready --sort priority works. Empty SortPolicy defaults to hybrid (backwards compatible).


Implement exclusive lock protocol for daemon/external tool coexistence

VC (VibeCoder) cannot coexist with bd daemon managing the same database. VC needs deterministic execution and directly manages JSONL sync, but daemon's auto-flush creates timing conflicts and nondeterministic behavior.

Current workaround (vc-195): VC kills ALL bd daemon processes with pkill, which is too aggressive and prevents developers from using daemon for their own workflow.

Solution: Implement lock file protocol (.beads/.exclusive-lock) that allows applications to claim exclusive management of a database. bd daemon respects these locks and skips locked databases.

See VC_DAEMON_EXCLUSION_PROTOCOL.md for full specification.

Lock File Protocol:
- File: .beads/.exclusive-lock (JSON format)
- Contains: holder name, PID, hostname, timestamp, version
- Created when external tool (VC) starts
- Removed on clean shutdown
- Stale lock detection: if PID doesn't exist, lock is removed

bd daemon behavior:
1. Before touching any database, check for .exclusive-lock
2. If lock exists and valid (PID alive): skip database in sync cycle
3. If lock is stale (PID dead): remove lock and proceed
4. If lock is malformed: log warning and skip (fail-safe)

External tool behavior:
1. On startup: create .exclusive-lock with process info
2. If lock exists, check if stale; fail if another process is running
3. On shutdown: remove .exclusive-lock

Benefits:
- Clean separation, no process killing
- Developer-friendly (can run daemon for other projects)
- Fail-safe with stale lock detection
- Extensible for other tools beyond VC

1. bd daemon skips databases with valid exclusive locks
2. bd daemon cleans up stale locks automatically
3. Lock file format documented and implemented
4. Integration tests show daemon + external tool coexistence
5. Documentation updated with protocol specification


Add ExclusiveLock struct and JSON marshaling

Create Go struct to represent exclusive lock file format with JSON marshaling support.

Fields needed:
- Holder (string): name of lock holder (e.g., "vc-executor")
- PID (int): process ID
- Hostname (string): hostname where process is running
- StartedAt (time.Time): when lock was acquired
- Version (string): version of lock holder

Should support both Marshal and Unmarshal for reading/writing lock files.


Implement isProcessAlive() helper for PID validation

Implement helper function to check if a process is alive given PID and hostname.

Logic:
- If hostname != current host: return true (can't verify remote, assume alive)
- If hostname == current host: check if PID exists using os.FindProcess + Signal(0)
- Return true if process exists, false otherwise

Needed for stale lock detection.


Implement shouldSkipDatabase() to check for exclusive locks

Add shouldSkipDatabase() function to daemon that checks for .beads/.exclusive-lock before processing a database.

Logic:
1. Check if .beads/.exclusive-lock exists
2. If not: return false (proceed with database)
3. If yes: read and parse JSON
4. If malformed: log warning and skip (fail-safe)
5. If valid: check if PID is alive
6. If stale (PID dead): remove lock file and proceed
7. If active (PID alive): log skip message and return true

Depends on ExclusiveLock struct and isProcessAlive() helper.


Integrate exclusive lock check into daemon sync cycle

Integrate shouldSkipDatabase() into daemon's sync cycle to respect exclusive locks.

Before processing each database:
1. Call shouldSkipDatabase(beadsDir)
2. If true: skip all operations (export, import, commit, push)
3. If false: proceed with normal sync

Add appropriate logging:
- "Skipping .beads/foo.db (locked by vc-executor, PID 12345)" when skipping
- "Removed stale lock at .beads/foo.db (PID 12345 not found)" when cleaning up

Depends on shouldSkipDatabase() implementation.


Phase 1: Add Beads Dependency (Non-Breaking)

Introduce Beads library alongside existing SQLite code without breaking production.

**Goal:** Add Beads as optional dependency with feature flag, establish compatibility baseline.

**Key Tasks:**
1. Add github.com/steveyegge/beads to go.mod
2. Create compatibility test suite comparing VC SQLite vs Beads schema
3. Identify schema differences and document migration requirements
4. Create internal/storage/beads/adapter.go implementing Storage interface
5. Add feature flag: VC_USE_BEADS_LIBRARY=true (disabled by default)

**Acceptance Criteria:**
- Beads library imported successfully
- Compatibility tests pass identifying all schema differences
- Both implementations coexist without conflicts
- No production impact (feature flag disabled by default)
- Documentation of schema differences and migration needs

**Technical Details:**
- Use feature flag to allow A/B testing
- Compatibility tests must cover: issues, dependencies, labels, status transitions, ID generation
- Adapter must implement full Storage interface
- Zero changes to existing production code paths

**Blockers:** None - can start immediately

**Estimated Effort:** 1 sprint


Add unit tests for exclusive lock detection

Add unit tests for exclusive lock functionality:

Test cases:
1. No lock file exists -> shouldSkipDatabase returns false
2. Valid lock with alive PID -> shouldSkipDatabase returns true
3. Stale lock with dead PID -> removes lock, returns false
4. Malformed lock JSON -> logs warning, returns true (fail-safe)
5. Lock with different hostname -> assumes alive, returns true
6. isProcessAlive() correctly detects running/dead processes

Use test helpers to create lock files with specific PIDs.


Add integration tests for daemon + external tool coexistence

Add integration tests simulating real-world daemon + external tool usage:

Test scenarios:
1. Start daemon, create lock file, verify daemon skips database
2. Create stale lock (with dead PID), start daemon, verify lock cleanup
3. Multiple lock/unlock cycles
4. Daemon resumes managing database after lock is removed
5. Lock file created/removed during daemon operation

Should verify:
- Daemon log messages correct
- No JSONL auto-flush when database is locked
- Daemon resumes normal operation after lock removed


Document exclusive lock protocol in README and new doc file

Update documentation to explain exclusive lock protocol:

1. Create EXCLUSIVE_LOCK.md with:
   - Protocol specification
   - Lock file format (JSON schema)
   - Usage examples for external tools
   - Stale lock behavior
   - Edge cases and limitations

2. Update README.md:
   - Mention exclusive lock support in daemon section
   - Link to EXCLUSIVE_LOCK.md for details

3. Update AGENTS.md:
   - Note that external tools can use exclusive locks
   - Explain daemon skips locked databases

Target audience: developers integrating bd with external tools (like VC).


Add staleness check to non-daemon mode

Extend staleness detection to non-daemon mode (--no-daemon).

Implementation:
- On database open, check if .beads/issues.jsonl exists
- If JSONL exists and is newer than .db file: auto-import
- Compare JSONL mtime vs .db mtime (both os.Stat)
- Log: "Auto-importing from .beads/issues.jsonl (newer than database)"

This ensures both daemon and non-daemon modes handle git pull correctly.


Add 'bd sync' command for explicit synchronization

Add explicit `bd sync` command as fallback for manual synchronization after git pull.

Behavior:
- Import from .beads/issues.jsonl
- If daemon mode: send RPC command to daemon to re-import
- If non-daemon: directly import to local db
- Show summary: "Imported N issues, updated M issues"

Usage:
```bash
git pull
bd sync  # Force immediate sync
```

This complements auto-detection but gives users manual control.


Add integration test for git pull sync scenario

Add integration test simulating the git pull sync issue.

Test scenario:
1. Create temp git repo with beads initialized
2. Clone 1: Create and close issue, export, commit, push
3. Clone 2: Start daemon, git pull
4. Clone 2: Verify bd show <issue> reflects closed status immediately
5. Verify no manual import or daemon restart needed

Also test:
- Non-daemon mode (--no-daemon) handles git pull correctly
- bd sync command works in both modes
- Performance: staleness check adds <10ms overhead

Depends on staleness detection implementation.


Add optional post-merge git hook example for bd sync

Create example git hook that auto-runs bd sync after git pull/merge.

Add to examples/git-hooks/:
- post-merge hook that checks if .beads/issues.jsonl changed
- If changed: run `bd sync` automatically
- Make it optional/documented (not auto-installed)

Benefits:
- Zero-friction sync after git pull
- Complements auto-detection as belt-and-suspenders

Note: post-merge hook already exists for pre-commit/post-merge. Extend it to support sync.


Update documentation for auto-sync behavior

Update documentation to explain auto-sync after git pull.

Files to update:
1. README.md - Add section on git workflow and auto-sync
2. AGENTS.md - Note that bd auto-detects JSONL changes after git pull
3. WORKFLOW.md - Update git pull workflow to remove manual import step
4. FAQ.md - Add Q&A about sync behavior and staleness

Key points:
- bd automatically detects when JSONL is newer than database
- No manual import needed after git pull
- bd sync command available for manual control
- Optional git hook for guaranteed sync


Refactor autoImportIfNewer to be callable from daemon

The staleness check in [deleted:bd-384] detects when JSONL is newer than last import, but can't trigger the actual import because autoImportIfNewer() is in cmd/bd and uses global variables.

Need to:
1. Extract core import logic from autoImportIfNewer() into importable function
2. Move to internal/autoimport or similar package
3. Make it callable from daemon (no global state dependency)
4. Update staleness check in server.go to call actual import instead of just logging

This completes the auto-sync feature - daemon will truly auto-import after git pull.


Enforce daemon singleton per workspace with file locking

Agent in ~/src/wyvern discovered 4 simultaneous daemon processes running, causing infinite directory recursion (.beads/.beads/.beads/...). Each daemon used relative paths and created nested .beads/ directories.

Root cause: No singleton enforcement. Multiple `bd daemon` processes can start in same workspace.

Expected: One daemon per workspace (each workspace = separate .beads/ dir with bd.sock)
Actual: Multiple daemons can run simultaneously in same workspace

Note: Separate git clones = separate workspaces = separate daemons (correct). Git worktrees share .beads/ and have known limitations (documented, use --no-daemon).

Use flock (file locking) on daemon socket or database file to enforce singleton:

1. On daemon start, attempt exclusive lock on .beads/bd.sock or .beads/daemon.lock
2. If lock held by another process, refuse to start (exit with clear error)
3. Hold lock for lifetime of daemon process
4. Release lock on daemon shutdown

Alternative: Use PID file with stale detection (check if PID is still running)

Implementation location: Daemon startup code in cmd/bd/ or internal/daemon/

1. Starting second daemon process in same workspace fails with clear error
2. Test: Start daemon, attempt second start, verify failure
3. Killing daemon releases lock, allowing new daemon to start
4. No infinite .beads/ directory recursion possible
5. Works correctly with auto-start mechanism


Phase 2: Implement VCStorage Wrapper

Create VCStorage wrapper that embeds beads.Storage and adds VC-specific operations.

**Goal:** Build clean abstraction layer where VC extends Beads without modifying Beads library.

**Architecture:**
- VCStorage embeds beads.Storage (delegates core operations)
- VCStorage adds VC-specific methods (executor instances, events)
- Same database, separate table namespaces (Beads tables + VC tables)
- Zero changes to Beads library code

**Key Tasks:**
1. Create VCStorage struct that embeds beads.Storage
2. Implement VC-specific methods: CreateExecutorInstance(), GetStaleExecutors(), LogEvent(), UpdateExecutionState()
3. Create VC table schemas (executor_instances, issue_execution_state, agent_events)
4. Verify type compatibility between VC types.Issue and Beads Issue
5. Create MockVCStorage for testing
6. Write unit tests for VC-specific methods
7. Write integration tests (end-to-end with Beads)
8. Benchmark performance vs current SQLite
9. Verify NO changes needed to Beads library

**Acceptance Criteria:**
- VCStorage successfully wraps Beads storage (embedding works)
- VC-specific tables created and accessible via foreign keys to Beads tables
- VC-specific methods work (executor instances, events)
- Core operations delegate to Beads correctly
- Tests pass with >90% coverage
- Performance benchmark shows no regression
- Beads library remains unmodified and standalone

**Technical Details:**
- Use beadsStore.DB() to get underlying database connection
- Create VC tables with FOREIGN KEY references to Beads issues table
- Schema separation: Beads owns (issues, dependencies, labels), VC owns (executor_instances, agent_events)
- Testing: Embed MockBeadsStorage in MockVCStorage

**Dependencies:**
- Blocked by Phase 1 (need Beads library imported)

**Estimated Effort:** 1.5 sprints


Track last JSONL import timestamp in daemon state

Add state tracking to daemon to record when .beads/issues.jsonl was last imported.

Implementation:
- Add lastImportTime field to daemon state (time.Time)
- Update timestamp after successful import
- Persist across RPC requests (in-memory state)
- Initialize on daemon startup

Needed for staleness detection to compare against JSONL mtime.


Implement staleness check before serving daemon requests

Add staleness detection to daemon RPC handler that checks if JSONL is newer than last import.

Logic:
1. Before processing any RPC request (show, list, ready, etc.)
2. Get .beads/issues.jsonl mtime (os.Stat)
3. Compare with lastImportTime
4. If JSONL mtime > lastImportTime: auto-import before processing request
5. If import fails: log error but continue with existing data

Performance: mtime check is fast (<1ms). Only imports if needed.

Depends on lastImportTime tracking.


Daemon fails to auto-import after git pull updates JSONL

After git pull updates .beads/issues.jsonl, daemon doesn't automatically re-import changes, causing stale data to be shown until next sync cycle (up to 5 minutes).

Reproduction:
1. Repo A: Close issues, export, commit, push
2. Repo B: git pull (successfully updates .beads/issues.jsonl)
3. bd show <issue> shows OLD status from daemon's SQLite db
4. JSONL on disk has correct new status

Root cause: Daemon sync cycle runs on timer (5min). When user manually runs git pull, daemon doesn't detect JSONL was updated externally and continues serving stale data from SQLite.

Impact:
- High for AI agents using beads in git workflows
- Breaks fundamental git-as-source-of-truth model
- Confusing UX: git log shows commit, bd shows old state
- Data consistency issues between JSONL and daemon

See WYVERN_SYNC_ISSUE.md for full analysis.

Three possible solutions:

Option 1: Auto-detect and re-import (recommended)
- Before serving any bd command, check if .beads/issues.jsonl mtime > last import time
- If newer, auto-import before processing request
- Fast check, minimal overhead

Option 2: File watcher in daemon
- Daemon watches .beads/issues.jsonl for mtime changes
- Auto-imports when file changes
- More complex, requires file watching infrastructure

Option 3: Explicit sync command
- User runs `bd sync` after git pull
- Manual, error-prone, defeats automation

Recommended: Option 1 (auto-detect) + Option 3 (explicit sync) as fallback.

1. After git pull updates .beads/issues.jsonl, next bd command sees fresh data
2. No manual import or daemon restart required
3. Performance impact < 10ms per command (mtime check is fast)
4. Works in both daemon and non-daemon modes
5. Test: Two repo clones, update in one, pull in other, verify immediate sync


Implement request-scoped routing with ContextVar

Add ContextVar-based routing to avoid global state races during concurrent multi-project calls.

Approach:
- Define current_workspace: ContextVar[str|None] in server.py
- Add @with_workspace decorator that resolves workspace_root (via _resolve_workspace_root + realpath)
- Set ContextVar for duration of tool call, reset after
- Falls back to set_context default (BEADS_WORKING_DIR) if workspace_root not provided
- beads_mcp.tools.get_client() reads current_workspace from ContextVar

Blocks: bd-135 (connection manager must exist first)

Decorator pattern with ContextVar for request-scoped workspace routing.

@with_workspace decorator:
- Extract workspace_root parameter from tool call
- Resolve via _resolve_workspace_root + realpath
- Set current_workspace ContextVar for request duration
- Falls back to BEADS_WORKING_DIR if workspace_root not provided
- Reset ContextVar after tool completes

Applied to all tools in server.py. _get_client() reads current_workspace.

⚠ CONCURRENCY GOTCHA (from architectural review):
- ContextVar doesn't propagate to asyncio.create_task() spawned tasks
- SOLUTION: Keep tool calls synchronous, no background task spawning
- If background tasks needed: use contextvars.copy_context()

Document this limitation in bd-141.

Blocks on bd-139 (connection pool must exist first).

CRITICAL: Do NOT spawn background tasks within tool implementations.
ContextVar propagation to spawned tasks is unreliable.


Add health checks and reconnection logic for stale sockets

Handle stale/broken socket connections after daemon restarts or upgrades.

Features:
- ping/health_check method on client
- Check before use or periodic health check
- On failure: drop from pool, attempt reconnect with exponential backoff
- Clear error propagation if reconnect fails after retries
- Handle version mismatch after daemon upgrade
- Handle long-idle connections closed by daemon


Fix require_context to support per-request workspace_root

Update require_context decorator to pass if either:
- workspace_root was provided on the tool call (via ContextVar), OR
- BEADS_WORKING_DIR is set (from set_context)

Stop using BEADS_DB as router - treat it only as CLI fallback.

Ensures backward compatibility with set_context() while supporting new per-request routing.


Add connection manager for per-project daemon sockets

Create connection pool for managing multiple daemon socket connections in beads_mcp.tools.

SIMPLIFIED APPROACH (per architectural review):
- Connection pool dict[str, BdClientBase] keyed by canonical path
- Add asyncio.Lock for thread-safe pool access (CRITICAL)
- Path canonicalization with @lru_cache for performance
- NO LRU eviction (start simple, add if monitoring shows memory issues)
- NO preemptive health checks (only retry on failure)

Features:
- get_client(workspace_root) → returns client for that project
- Canonicalization: realpath + git toplevel (handles symlinks)
- Submodule-aware: Check local .beads BEFORE git toplevel
- Auto-start daemon if socket missing (existing behavior)
- Validate version on connect (existing behavior)

CRITICAL: Replace global _client singleton with pool to fix set_context() bug.


Phase 3: Migration Path & Database Schema Alignment

Enable existing .beads/vc.db files to work with Beads library through automated migration.

**Goal:** Provide safe, tested migration path from SQLite implementation to Beads library.

**Key Tasks:**
1. Run compatibility tests against production databases
2. Identify schema differences (columns, indexes, constraints)
3. Document required migrations
4. Create migration CLI command: 'vc migrate --from sqlite --to beads'
5. Add dry-run mode for preview
6. Add backup/restore capability
7. Implement rollback mechanism
8. Add auto-detection of schema version on startup
9. Add auto-migrate with user prompt

**Acceptance Criteria:**
- Existing databases migrate successfully
- Data integrity preserved (zero data loss verified via checksums)
- Rollback works if migration fails
- Migration tested on real production VC databases
- Dry-run mode shows exactly what will change
- Backup created before migration
- Feature flag: VC_FORCE_SQLITE=true provides escape hatch

**Technical Details:**
- Compare current SQLite schema with Beads schema
- Handle version detection (read schema_version or detect from structure)
- Migration should be idempotent (safe to run multiple times)
- Backup strategy: Copy .beads/vc.db to .beads/vc.db.backup-<timestamp>
- Verify foreign key integrity after migration

**Safety Measures:**
- Require executor shutdown before migration (check for running executors)
- Atomic migration (BEGIN IMMEDIATE transaction)
- Comprehensive pre/post migration validation
- Clear error messages with recovery instructions

**Dependencies:**
- Blocked by Phase 2 (need VCStorage implementation)

**Estimated Effort:** 0.5 sprint


Add integration tests for multi-project MCP switching

Comprehensive tests to verify multi-project isolation, concurrency, and edge cases.

EXPANDED TEST COVERAGE (per architectural review):

**Concurrency tests (CRITICAL):**
- asyncio.gather() with calls to different workspace_root values
- Verify no cross-project data leakage
- Verify pool lock prevents race conditions

**Edge case tests:**
- Submodule handling: Parent repo vs submodule with own .beads
- Symlink deduplication: Same physical path via different symlinks
- Stale socket recovery: Kill daemon, verify retry on failure
- Missing .beads directory handling

**Isolation tests:**
- Create 2+ temp repos with bd init
- Verify operations in project A don't affect project B
- Stress test: many parallel calls across 3-5 repos

Estimated effort: M (1-2 days) including fixtures for temp repos and daemon process management


Update MCP multi-project documentation

Update documentation for multi-project workflow in README.md, AGENTS.md, and MCP integration docs.

EXPANDED SECTIONS (per architectural review):

**Usage examples:**
- Per-request workspace_root parameter usage
- Concurrent multi-project queries with asyncio.gather
- Migration from set_context() to workspace_root parameter

**Architecture notes:**
- Connection pooling behavior (no limits initially)
- set_context() as default fallback (still supported)
- Library users NOT affected (all changes in MCP layer)

**Concurrency gotchas (CRITICAL):**
- ContextVar doesn't propagate to asyncio.create_task()
- Do NOT spawn background tasks in tool implementations
- All tool calls should be synchronous/sequential

**Troubleshooting:**
- Stale sockets (retry once on failure)
- Version mismatches (auto-detected since v0.16.0)
- Path aliasing via symlinks (deduplicated by realpath)
- Submodules with own .beads (handled correctly)

**Use cases:**
- Multi-organization collaboration (GH#145)
- Parallel project management scripts
- Cross-project queries


⚠ DEFERRED TO PHASE 2 (per architectural review)

Add health checks and reconnection logic for stale sockets ONLY IF monitoring shows it's needed.

SIMPLIFIED APPROACH:
- NO preemptive health checks (adds latency to every call)
- NO periodic ping before use (daemon restarts are rare)
- YES: Single retry on connection failure (DaemonConnectionError)
- YES: Evict stale client from pool on failure

Rationale:
- Stale sockets are rare (daemon auto-restart is uncommon)
- Preemptive checks add latency with little benefit
- Retry-on-failure is sufficient for most cases

Implementation (if needed later):
- Wrap tool calls with try/except
- On DaemonConnectionError: evict from pool, retry once
- Log failures for monitoring

Monitor after Phase 1 launch:
- Frequency of stale socket errors
- User reports of connection issues
- Decision point: Add if >1% of calls fail

Simplified retry wrapper (implement only if monitoring shows need):

```python
async def _robust_client_call(func):
    try:
        client = await _get_client()
        return await func(client)
    except (DaemonConnectionError, DaemonNotRunningError):
        # Evict stale client and retry once
        workspace = current_workspace.get()
        canonical = _canonicalize_path(workspace)
        async with _pool_lock:
            _connection_pool.pop(canonical, None)
        # Retry
        client = await _get_client()
        return await func(client)
```

NO bounded backoff, NO health checks, NO version validation pings.

DEFERRED - Not needed for MVP.

Add to Phase 2 roadmap only if monitoring shows:
- Stale socket errors >1% of calls
- User complaints about connection issues
- Long-running MCP servers experiencing problems


bd daemon auto-sync can wipe out issues.jsonl when database is empty

During dogfooding session, bd daemon auto-sync exported empty database to JSONL, losing all 177 issues. Had to git restore to recover.

Root cause: bd export doesn't check if database is empty before exporting. When daemon has empty/wrong database, it wipes out valid JSONL file.

Impact: DATA LOSS


bd import doesn't update database modification time (WAL mode)

When running bd import in WAL mode, the -wal file is updated but main .db file timestamp stays old. This breaks staleness detection which only checks main .db file.

Discovered during dogfooding when import didn't trigger staleness refresh.

Impact: Staleness checks fail to detect that database is newer than expected


bd should show which database file it's using

During dogfooding, bd showed "0 issues" when correct database had 177 issues. Confusion arose from which database path was being used (daemon default vs explicit --db flag).

Users need clear feedback about which database file bd is actually using, especially when daemon is involved.

Impact: User confusion, working with wrong database unknowingly


Add configurable SortPolicy to GetReadyWork

Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.


Implement configurable sort policy for GetReadyWork

Add SortPolicy field to WorkFilter to support different sorting strategies:
- hybrid (default): Recent issues by priority, old by age
- priority: Strict priority ordering for autonomous execution
- oldest: Backlog clearing mode

Solves issue where VC executor selects low-priority work instead of critical P0 work.


Make merge command idempotent for safe retry after partial failures

The merge command currently performs 3 operations without an outer transaction:
1. Migrate dependencies from source → target
2. Update text references across all issues
3. Close source issues

If merge fails mid-operation (network issue, daemon crash, etc.), a retry will fail or produce incorrect results because some operations already succeeded.

**Goal:** Make merge idempotent so retrying after partial failure is safe and completes the remaining work.

**Idempotency checks needed:**
- Skip dependency migration if target already has the dependency
- Skip text reference updates if already updated
- Skip closing source issues if already closed
- Report which operations were skipped vs performed

**Example output:**
```
✓ Merged 2 issue(s) into bd-227
  - Dependencies: 3 migrated, 2 already existed
  - Text references: 5 updated, 0 already correct
  - Source issues: 1 closed, 1 already closed
```

**Related:** bd-219 originally requested transaction support, but idempotency is a better solution for this use case since individual operations are already atomic.

Current merge code already has some idempotency:
- Dependency migration checks `alreadyExists` before adding (line ~145-151 in merge.go)
- Text reference updates are naturally idempotent (replacing bd-X with bd-Y twice has same result)

Missing idempotency:
- CloseIssue fails if source already closed
- Error messages don't distinguish "already done" from "real failure"

Implementation:
1. Check source issue status before closing - skip if already closed
2. Track which operations succeeded/skipped
3. Return detailed results for user visibility
4. Consider adding --dry-run output showing what would be done vs skipped


The merge command currently performs 3 operations without an outer transaction:
1. Migrate dependencies from source → target
2. Update text references across all issues
3. Close source issues

If merge fails mid-operation (network issue, daemon crash, etc.), a retry will fail or produce incorrect results because some operations already succeeded.

**Goal:** Make merge idempotent so retrying after partial failure is safe and completes the remaining work.

**Idempotency checks needed:**
- Skip dependency migration if target already has the dependency
- Skip text reference updates if already updated
- Skip closing source issues if already closed
- Report which operations were skipped vs performed

**Example output:**
```
✓ Merged 2 issue(s) into bd-392
  - Dependencies: 3 migrated, 2 already existed
  - Text references: 5 updated, 0 already correct
  - Source issues: 1 closed, 1 already closed
```

**Related:** bd-398 originally requested transaction support, but idempotency is a better solution for this use case since individual operations are already atomic.


Phase 4: Gradual Cutover & Production Rollout

Replace SQLite implementation with Beads library in production and remove legacy code.

**Goal:** Complete transition to Beads library, deprecate and remove custom SQLite implementation.

**Key Tasks:**
1. Run VC executor with Beads library in CI
2. Dogfood: Use Beads library for VC's own development
3. Monitor for regressions and performance issues
4. Flip feature flag: VC_USE_BEADS_LIBRARY=true by default
5. Monitor production logs for errors
6. Collect user feedback
7. Add deprecation notice to CLAUDE.md
8. Provide migration guide for users
9. Remove legacy code: internal/storage/sqlite/sqlite.go (~1500 lines)
10. Remove migration framework: internal/storage/migrations/
11. Remove manual transaction management code
12. Update all documentation

**Acceptance Criteria:**
- Beads library enabled by default in production
- Zero production incidents related to migration
- Performance meets or exceeds SQLite implementation
- All tests passing with Beads library
- Legacy SQLite code removed
- Documentation updated
- Celebration documented 🎉

**Rollout Strategy:**
1. Week 1: Enable for CI/testing environments
2. Week 2: Dogfood on VC development
3. Week 3: Enable for 50% of production (canary)
4. Week 4: Enable for 100% of production
5. Week 5: Remove legacy code

**Monitoring:**
- Track error rates before/after cutover
- Monitor database query performance
- Track issue creation/update latency
- Monitor executor claim performance

**Rollback Plan:**
- Keep VC_FORCE_SQLITE=true escape hatch for 2 weeks post-cutover
- Keep legacy code for 1 sprint after cutover
- Document rollback procedure

**Success Metrics:**
- Zero data loss
- No performance regression (< 5% latency increase acceptable)
- Reduced maintenance burden (code LOC reduction)
- Positive developer feedback

**Dependencies:**
- Blocked by Phase 3 (need migration tooling)

**Estimated Effort:** 1 sprint


Counter not synced after import on existing DB with populated issue_counters table

The counter sync fix in counter_sync_test.go only syncs during initial migration when issue_counters table is empty (migrateIssueCountersTable checks count==0). For existing databases with stale counters:

- Import doesn't resync the counter
- Delete doesn't update counter
- Renumber doesn't fix counter
- Counter remains stuck at old high value

Example from today:
- Had 49 issues after clean import
- Counter stuck at 4106 from previous test pollution
- Next issue would be bd-4107 instead of bd-404
- Even after renumber, counter stayed at 4106

Root cause: Migration only syncs if table is empty (line 182 in sqlite.go). Once populated, never resyncs.

Fix needed:
1. Sync counter after import operations (not just empty table)
2. Add counter resync after renumber
3. Daemon caches counter value - needs to reload after external changes

Related: bd-394 (original counter sync fix), bd-399 (daemon cache staleness)

## Investigation Results

After thorough code review, all the fixes mentioned in the issue description have ALREADY been implemented:

### ✅ Fixes Already in Place:

1. **Import DOES resync counters**
   - `cmd/bd/import_shared.go:253` calls `SyncAllCounters()` after batch import
   - Verified with new test `TestCounterSyncAfterImport`

2. **Delete DOES update counters**
   - `internal/storage/sqlite/sqlite.go:1424` calls `SyncAllCounters()` after deletion
   - Both single delete and batch delete sync properly
   - Verified with existing tests: `TestCounterSyncAfterDelete`, `TestCounterSyncAfterBatchDelete`

3. **Renumber DOES fix counters**
   - `cmd/bd/renumber.go:298-304` calls `ResetCounter()` then `SyncAllCounters()`
   - Forces counter to actual max ID (not just MAX with stale value)

4. **Daemon cache DOES detect external changes**
   - `internal/rpc/server.go:1466-1487` checks file mtime and evicts stale cache
   - When DB file changes externally, cached storage is evicted and reopened

### Tests Added:

- `TestCounterSyncAfterImport`: Confirms import syncs counters from stale value (4106) to actual max (49)
- `TestCounterNotSyncedWithoutExplicitSync`: Documents what would happen without the fix (bd-4107 instead of bd-404)

### Conclusion:

The issue described in bd-404 has been **fully resolved**. All operations (import, delete, renumber) now properly sync counters. The daemon correctly detects external DB changes via file modification time.

The root cause (migration only syncing empty tables) was fixed by adding explicit `SyncAllCounters()` calls after import, delete, and renumber operations.


Add godoc comments for auto-flush functions

Add comprehensive godoc comments for findJSONLPath(), markDirtyAndScheduleFlush(), and flushToJSONL() explaining behavior, concurrency considerations, and error handling. Include notes about debouncing behavior (timer resets on each write, flush occurs 5s after LAST operation) and flush-on-exit guarantees. Located in cmd/bd/main.go:188-307.


Document merge command and AI integration

Update README, AGENTS.md with merge command examples. Document AI agent duplicate detection workflow.


Stress tests pollute production database with test issues

TestStressNoUniqueConstraintViolations and other stress tests in internal/rpc/stress_test.go create issues in production database instead of test database. Confirmed: 1,000 test issues (Agent X Issue Y) created at 20:46:01 during test run. Root cause: test goroutines connect to production daemon at .beads/bd.sock instead of isolated test daemon in temp directory.


Add --id flag to bd list for filtering by specific issue IDs

Add --id flag accepting comma-separated IDs. Usage: bd list --id wy-11,wy-12. Combines with other filters. From filter-flag-design.md.


Add customizable time threshold for compact command

Currently compact uses fixed 30-day and 90-day tiers. Add support for custom time thresholds like '--older-than 60h' or '--older-than 2.5d' to allow more flexible compaction policies.

Examples:
  bd compact --all --older-than 60h
  bd compact --all --older-than 2.5d
  bd compact --all --tier 1 --age 48h

This would allow users to set their own compaction schedules based on their workflow needs.


Consider implementing pre-commit hooks for Storage interface changes

The documentation (INTERFACE_CHANGES.md) suggests adding pre-commit hooks that automatically check for Storage interface changes and verify all mocks are updated. This would prevent similar issues in the future where interface changes break mock implementations.

Discovered during execution of vc-228 (dogfooding run #14/15).

Implement a pre-commit hook that:
1. Detects changes to internal/storage/storage.go
2. Runs scripts/find-storage-mocks.sh to find all mock implementations
3. Attempts to compile all test files with mocks
4. Blocks commit if compilation fails

Tools: husky, pre-commit framework, or simple .git/hooks/pre-commit script

- Pre-commit hook installed and documented
- Hook detects Storage interface changes
- Hook validates all mocks compile
- Hook can be bypassed with --no-verify if needed
- Documentation updated with installation instructions


Test auto-export timing


Investigate stress test database pollution (vc-248)

Investigation of stress tests polluting production database with 1,600+ test issues on Oct 21 at 20:24-20:25. Root cause analysis completed. Tests now verified to work correctly with proper isolation.

Bug confirmed! Tests DO pollute production DB. 1,000 test issues created at 20:46:01-20:46:02 during TestStressNoUniqueConstraintViolations. Root cause: test goroutines connect to production daemon at .beads/bd.sock instead of test daemon.


Compact command fails with daemon - requires --no-daemon workaround

The 'bd compact' command fails with 'Error: compact requires SQLite storage' when used with the daemon (default mode), but works correctly with the '--no-daemon' flag.

The daemon RPC interface doesn't properly expose the compact command, even though the daemon itself uses SQLite storage.

Reproduction:
1. Ensure daemon is running (bd daemon status)
2. Run: bd compact --stats
   Result: Error: compact requires SQLite storage
3. Run: bd compact --stats --no-daemon
   Result: Works correctly, shows statistics

Expected behavior:
The compact command should work through the daemon RPC interface just like other commands (list, create, update, delete, renumber, etc.)

Impact:
Users cannot use compact operations in the normal workflow. They must use --no-daemon which bypasses the daemon entirely.

Suggested fix:
Add compact operation support to the daemon RPC interface, similar to how renumber and other operations are exposed.


Add lifecycle safety docs and tests for UnderlyingDB() method

The new UnderlyingDB() method exposes the raw *sql.DB connection for extensions like VC to create their own tables. While database/sql is concurrency-safe, there are lifecycle and misuse risks that need documentation and testing.

**What needs to be done:**

1. **Enhanced documentation** - Expand UnderlyingDB() comments to warn:
   - Callers MUST NOT call Close() on returned DB
   - Do NOT change pool/driver settings (SetMaxOpenConns, SetConnMaxIdleTime)
   - Do NOT modify SQLite PRAGMAs (WAL mode, journal, etc.)
   - Expect errors after Storage.Close() - use contexts
   - Keep write transactions short to avoid blocking core storage

2. **Add lifecycle tracking** - Implement closed flag:
   - Add atomic.Bool closed field to SQLiteStorage
   - Set flag in Close(), clear in New()
   - Optional: Add IsClosed() bool method

3. **Add safety tests** (run with -race):
   - TestUnderlyingDB_ConcurrentAccess - N goroutines using UnderlyingDB() during normal storage ops
   - TestUnderlyingDB_AfterClose - Verify operations fail cleanly after storage closed
   - TestUnderlyingDB_CreateExtensionTables - Create VC table with FK to issues, verify FK enforcement
   - TestUnderlyingDB_LongTxDoesNotCorrupt - Ensure long read tx doesn't block writes indefinitely

**Why this matters:**
VC will use this to create tables in the same database. Need to ensure production-ready safety without over-engineering.

**Estimated effort:** S+S+S = M total (1-3h)

Oracle recommends "simple path": enhanced docs + minimal guardrails + focused tests. See oracle output for detailed rationale on concurrency safety, lifecycle risks, and when to consider advanced path (wrapping interface).


Fix flaky CI tests in compactor and daemon modules

Multiple test failures observed in CI:
- TestCompactTier1_DryRun
- TestCompactTier1Batch_DryRun
- TestCompactTier1Batch_WithIneligible
- TestMockAPI_CompactTier1
- TestBatchOperations_ErrorHandling
- TestCommentOperationsViaRPC
- TestMemoryPressureDetection
- TestPing

All failures related to 'issue has open dependents or not closed long enough' eligibility checks.

See CI run: https://github.com/steveyegge/beads/actions/runs/18688772658

Fixed race condition in compactor_test.go. The createClosedIssue helper was setting ClosedAt to time.Now(), but the eligibility check uses '<= datetime(now, -0 days)'. Changed to now.Add(-1*time.Second) to ensure issues are always eligible. All tests now passing.


Implement dependency migration for merge

Migrate all dependencies from source issue(s) to target issue during merge, removing duplicates and preserving graph integrity


Add transaction support for atomic merges

Wrap all merge operations in SQLite transaction for atomicity. Implement rollback on failure.


Document label best practices and use cases

Create documentation covering:
- When to use labels vs structured fields
- Common label sets (coding agents, open source, product dev, SRE)
- Naming conventions (kebab-case, specificity, present tense)
- Anti-patterns (too many labels, overlapping, personal labels)
- Label lifecycle management

Content from LABELS.md analysis document.


Add CLI merge command and flags

Implement bd merge command with: multiple sources, --into target, --dry-run, --json flags. Add interactive confirmation.


Implement text reference scanning and replacement

Scan all issues for text references to merged IDs (bd-X patterns) and update to target ID. Reuse logic from import collision resolution.


Update export/import for merge fields

Include merged_into in JSONL export format. Handle merge relationships on import.


Auto-flush writes test pollution and session work to git-tracked issues.jsonl

Auto-flush exports ALL issues from DB to issues.jsonl every 5 seconds, including:
- Test issues (bd-4053 through bd-4059 were version test junk)
- Issues created during debugging sessions
- Test pollution from stress tests
- Temporary diagnostic issues

This pollutes the git-tracked issues.jsonl with garbage that shouldn't be committed.

Example from today:
- Git had 49 clean issues
- Our DB grew to 100+ with test junk and session work
- Auto-flush wrote all 100+ to issues.jsonl
- Git status showed modified issues.jsonl with 50+ unwanted issues

Impact:
- Pollutes git history with test/debug garbage
- Makes code review difficult (noise in diffs)
- Can't distinguish real work from session artifacts
- Other team members pull polluted issues

Solutions to consider:
1. Disable auto-flush by default (require explicit --enable-auto-flush)
2. Add .beadsignore to exclude issue ID patterns
3. Make auto-flush only export 'real' issues (exclude test-*)
4. Require manual 'bd sync' for git commit
5. Auto-flush to separate file (.beads/session.jsonl vs issues.jsonl)

Related: bd-417 (test pollution), isolation_test.go (test DB separation)


Issue counter gets out of sync with actual issues

The issue counter in issue_counters table frequently desyncs from actual max issue ID, causing:
- Import from JSONL leaves counter at old high value
- Test pollution increments counter but cleanup doesn't decrement it
- Delete issues doesn't update counter
- Only fix is 'rm -rf .beads' which is destructive

Examples from today's session:
- Had 48 issues but counter at 7714 after test pollution
- Import from git didn't reset counter
- Next new issue would be bd-7715 instead of bd-417

Proposed fixes:
1. Auto-recalculate counter from max(issue_id) on import
2. Add 'bd fix-counter' command
3. Make counter lazy (always compute from DB, don't store)
4. Import should reset counter to match imported data

Related:-254 (test isolation), bd-414 (init timestamp bug)


Counter not synced after import on existing DB with populated issue_counters table

The counter sync fix in counter_sync_test.go only syncs during initial migration when issue_counters table is empty (migrateIssueCountersTable checks count==0). For existing databases with stale counters:

- Import doesn't resync the counter
- Delete doesn't update counter
- Renumber doesn't fix counter
- Counter remains stuck at old high value

Example from today:
- Had 49 issues after clean import
- Counter stuck at 4106 from previous test pollution
- Next issue would be bd-4107 instead of bd-414
- Even after renumber, counter stayed at 4106

Root cause: Migration only syncs if table is empty (line 182 in sqlite.go). Once populated, never resyncs.

Fix needed:
1. Sync counter after import operations (not just empty table)
2. Add counter resync after renumber
3. Daemon caches counter value - needs to reload after external changes

Related: bd-417 (original counter sync fix), bd-382 (daemon cache staleness)

## Investigation Results

After thorough code review, all the fixes mentioned in the issue description have ALREADY been implemented:

### ✅ Fixes Already in Place:

1. **Import DOES resync counters**
   - `cmd/bd/import_shared.go:253` calls `SyncAllCounters()` after batch import
   - Verified with new test `TestCounterSyncAfterImport`

2. **Delete DOES update counters**
   - `internal/storage/sqlite/sqlite.go:1424` calls `SyncAllCounters()` after deletion
   - Both single delete and batch delete sync properly
   - Verified with existing tests: `TestCounterSyncAfterDelete`, `TestCounterSyncAfterBatchDelete`

3. **Renumber DOES fix counters**
   - `cmd/bd/renumber.go:298-304` calls `ResetCounter()` then `SyncAllCounters()`
   - Forces counter to actual max ID (not just MAX with stale value)

4. **Daemon cache DOES detect external changes**
   - `internal/rpc/server.go:1466-1487` checks file mtime and evicts stale cache
   - When DB file changes externally, cached storage is evicted and reopened

### Tests Added:

- `TestCounterSyncAfterImport`: Confirms import syncs counters from stale value (4106) to actual max (49)
- `TestCounterNotSyncedWithoutExplicitSync`: Documents what would happen without the fix (bd-4107 instead of bd-414)

### Conclusion:

The issue described in bd-414 has been **fully resolved**. All operations (import, delete, renumber) now properly sync counters. The daemon correctly detects external DB changes via file modification time.

The root cause (migration only syncing empty tables) was fixed by adding explicit `SyncAllCounters()` calls after import, delete, and renumber operations.


Update EXTENDING.md with UnderlyingDB() usage and best practices

EXTENDING.md currently shows how to use direct sql.Open() to access the database, but doesn't mention the new UnderlyingDB() method that's the recommended way for extensions.

**Update needed:**
1. Add section showing UnderlyingDB() usage:
   ```go
   store, err := beads.NewSQLiteStorage(dbPath)
   db := store.UnderlyingDB()
   // Create extension tables using db
   ```

2. Document when to use UnderlyingDB() vs direct sql.Open():
   - Use UnderlyingDB() when you want to share the storage connection
   - Use sql.Open() when you need independent connection management

3. Add safety warnings (cross-reference from UnderlyingDB() docs):
   - Don't close the DB
   - Don't modify pool settings
   - Keep transactions short

4. Update the VC example to show UnderlyingDB() pattern

5. Explain beads.Storage.UnderlyingDB() in the API section


Add merged_into field to database schema

Add merged_into field to Issue struct and update database schema to support merge tracking

Simplified: no schema field needed. Close merged issues with reason 'Merged into bd-X'. See bd-263 design.


Re-land TestDatabaseReinitialization after fixing Windows/Nix issues

TestDatabaseReinitialization test was reverted due to CI failures:
- Windows: JSON parse errors, missing files
- Nix: git not available in build environment

Need to fix and re-land:
1. Make test work on Windows (path separators, file handling)
2. Skip test in Nix environment or mock git
3. Fix JSON parsing issues


Feature: Use external_ref as primary matching key for import updates

Implement external_ref-based matching for imports to enable hybrid workflows with external systems (Jira, GitHub, Linear).

## Problem
Current import collision detection treats any content change as a collision, preventing users from syncing updates from external systems without creating duplicates.

## Solution
Use external_ref field as primary matching key during imports. When an incoming issue has external_ref set:
- Search for existing issue with same external_ref
- If found, UPDATE (not collision)
- If not found, create new issue
- Never match local issues (without external_ref)

## Use Cases
- Jira integration: Import backlog, add local tasks, re-sync updates
- GitHub integration: Import issues, track with local subtasks, sync status
- Linear integration: Team coordination with local breakdown

## Reference
GitHub issue #142: https://github.com/steveyegge/beads/issues/142


GH#146: No color showing in terminal for some users

User reports color not working in macOS (Taho 26.0.1) with iTerm 3.6.4 and Terminal.app, despite color working elsewhere in terminal. Python rich and printf escape codes work.

Need to investigate:
- Is NO_COLOR env var set?
- Terminal type detection?
- fatih/color library configuration
- Does bd list show colors? bd ready? bd init?
- What's the output of: echo $TERM, echo $NO_COLOR


Fix nil pointer crash in bd reopen command

bd reopen crashes with SIGSEGV at reopen.go:30. Nil pointer dereference when trying to reopen an issue.


Address gosec security warnings (102 issues)

Security linter warnings: file permissions (0755 should be 0750), G304 file inclusion via variable, G204 subprocess launches. Many are false positives but should be reviewed.

Review each gosec warning. Add exclusions for legitimate cases to .golangci.yml. Fix real security issues (overly permissive file modes).


Multi-project MCP context switching (GH#145)

Enable MCP server to manage multiple beads projects in a single session with per-request workspace_root parameter.

Current bug: set_context(workspace_root) doesn't actually switch sockets - all operations hit first initialized socket.

Use case: Managing tasks across multiple organizations with different permission models (internal, partners, open source).

Architecture: Connection pool keyed by workspace_root, each maintaining its own daemon socket connection. Request-scoped routing using ContextVar to avoid global state races.

See GH#145 for full requirements and user context.

✅ APPROVED WITH MODIFICATIONS by architectural review (2025-10-25)

LSP-style model is correct: Single MCP server → per-project daemons → isolated databases.

CRITICAL CHANGES from review:
1. **Simplify connection pool**: No LRU eviction initially (typical user: 2-5 projects)
2. **Add asyncio.Lock**: Prevent race conditions in pool access
3. **Defer health checks**: Only retry on failure, not preemptive pings
4. **Handle submodules**: Check local .beads BEFORE git toplevel
5. **Path canonicalization**: realpath + git toplevel with caching

RISKS MITIGATED:
- Global _client bug: Replace with connection pool keyed by canonical path
- Race conditions: Add asyncio.Lock for pool mutations
- Submodule edge case: Check .beads directory first
- Stale sockets: Retry once on connection failure

Estimated effort: 2.5-3.5 days (simplified from 2.5-4.5 days)
Confidence: 8/10

Review doc: docs/bd-391-architectural-review.md

Implementation order:
1. bd-139 (connection manager) - Foundation with pool + lock
2. bd-136 (ContextVar routing) - Per-request workspace
3. bd-138 (require_context) - Validation
4. bd-140 (tests) - Concurrency + edge cases
5. bd-141 (docs) - Usage guide
6. bd-142 (health checks) - DEFERRED to Phase 2


Auto-flush exports ALL issues from DB to issues.jsonl every 5 seconds, including:
- Test issues (bd-4053 through bd-4059 were version test junk)
- Issues created during debugging sessions
- Test pollution from stress tests
- Temporary diagnostic issues

This pollutes the git-tracked issues.jsonl with garbage that shouldn't be committed.

Example from today:
- Git had 49 clean issues
- Our DB grew to 100+ with test junk and session work
- Auto-flush wrote all 100+ to issues.jsonl
- Git status showed modified issues.jsonl with 50+ unwanted issues

Impact:
- Pollutes git history with test/debug garbage
- Makes code review difficult (noise in diffs)
- Can't distinguish real work from session artifacts
- Other team members pull polluted issues

Solutions to consider:
1. Disable auto-flush by default (require explicit --enable-auto-flush)
2. Add .beadsignore to exclude issue ID patterns
3. Make auto-flush only export 'real' issues (exclude test-*)
4. Require manual 'bd sync' for git commit
5. Auto-flush to separate file (.beads/session.jsonl vs issues.jsonl)

Related: bd-168 (test pollution), isolation_test.go (test DB separation)

## Analysis

Confirmed the issue exists - bd-167 through bd-182 are test pollution in the git-tracked issues.jsonl.

### Solution Evaluation:

**Option 1: Disable auto-flush by default** ❌
- Breaks the auto-sync workflow that users rely on
- Requires manual intervention which defeats the purpose
- Not recommended

**Option 2: Add .beadsignore** ⚠
- Complex to implement (pattern matching, configuration)
- Doesn't solve root cause: test issues in production DB
- Better to prevent pollution at source

**Option 3: Filter on export** ❌
- Doesn't solve root cause
- Test issues still pollute production DB
- Complicates export logic

**Option 4: Manual 'bd sync'** ❌
- Same issues as Option 1
- Breaks automated workflow

**Option 5: Separate session file** ❌
- Splits issue tracking across files
- Confusing for users
- Import/export complexity

### RECOMMENDED SOLUTION:

**Fix the root cause: Tests should NEVER touch the production database**

The real problem is that Go tests ARE properly isolated (they use temp DBs), but someone must be manually creating test issues in the production DB during development/debugging.

**Best fix:**
1. Document that production DB is for real work only
2. Add a convenience command: `bd test-create` that uses a separate test database
3. Clean up the existing test pollution: bd-167 through bd-182
4. Consider adding a git pre-commit hook to warn about suspicious issues

This preserves auto-flush (which is valuable) while preventing pollution at the source.

## Resolution

**Root Cause Identified:**
The issue was NOT a bug in auto-flush, but rather test pollution in the production database from manual testing/debugging. Go tests are properly isolated using temp directories.

**Actions Taken:**
1. Cleaned up test pollution: deleted bd-167 through bd-182 (all "Version test issue" entries)
2. Verified auto-flush is working correctly - it exports the database as designed
3. Confirmed Go test isolation works properly (uses temp dirs, not production DB)

**Prevention Strategy:**
- Production database (.beads/) should only contain real work issues
- Manual testing should use throwaway databases or test scripts
- Go tests already use isolated temp databases
- Auto-flush is working as intended and should remain enabled

**Conclusion:**
This was user error, not a system bug. The auto-flush mechanism is correct - it should export ALL database contents. The problem was polluting the production database with test issues in the first place.


Optimize auto-flush to use incremental updates

Every flush exports ALL issues and ALL dependencies, even if only one issue changed. For large projects (1000+ issues), this could be expensive. Current approach guarantees consistency, which is fine for MVP, but future optimization could track which issues changed and use incremental updates. Located in cmd/bd/main.go:255-276.


Warn when multiple beads databases detected in workspace hierarchy

Add detection and warning when multiple .beads directories exist in the workspace hierarchy to prevent confusion and database pollution.

## Problem
Users can accidentally work in the wrong beads database when multiple exist (e.g., ~/src/beads and ~/src/fred/beads), leading to:
- Database pollution from cross-contamination
- Lost work when changes go to wrong database
- Confusion about which database is active

## Solution
On startup, scan parent directories for additional .beads directories and:
- Warn if multiple found
- Show which database is currently active
- Suggest consolidation or cleanup

## Detection Strategy
1. Walk up directory tree from CWD
2. Check for .beads/bd.db in each level
3. If multiple found, show warning with paths
4. Optionally: check common sibling directories

## Example Warning
```
⚠  Multiple beads databases detected:
  - /Users/stevey/src/beads/.beads (ACTIVE - 111 issues)
  - /Users/stevey/src/fred/beads/.beads (202 issues)

Consider consolidating or removing unused databases.
```


Consider adding UnderlyingConn(ctx) for safer scoped DB access

Currently UnderlyingDB() returns *sql.DB which is correct for most uses, but for extension migrations/DDL, a scoped connection might be safer.

**Proposal:** Add optional UnderlyingConn(ctx) (*sql.Conn, error) method that:
- Returns a scoped connection via s.db.Conn(ctx)
- Encourages lifetime-bounded usage
- Reduces temptation to tune global pool settings
- Better for one-time DDL operations like CREATE TABLE

**Implementation:**
```go
// UnderlyingConn returns a single connection from the pool for scoped use
// Useful for migrations and DDL. Close the connection when done.
func (s *SQLiteStorage) UnderlyingConn(ctx context.Context) (*sql.Conn, error) {
    return s.db.Conn(ctx)
}
```

**Benefits:**
- Safer for migrations (explicit scope)
- Complements UnderlyingDB() for different use cases
- Low implementation cost

**Trade-off:** Adds another method to maintain, but Oracle considers this balanced compromise between safety and flexibility.

**Decision:** This is optional - evaluate based on VC's actual usage patterns.


Code review: external_ref import feature

Comprehensive code review of external_ref import implementation.

## Review Checklist
- [ ] Storage layer changes are correct and efficient
- [ ] Index created properly on external_ref column
- [ ] Collision detection logic handles all edge cases
- [ ] Import flow correctly distinguishes external vs local issues
- [ ] No breaking changes to existing workflows
- [ ] Tests cover all scenarios (see-109)
- [ ] Documentation is clear and complete (see-110)
- [ ] Performance impact is acceptable
- [ ] Error messages are helpful
- [ ] Logging is appropriate for debugging

## Oracle Review
Use oracle tool to analyze implementation for correctness, edge cases, and potential issues.


Comprehensive code review of external_ref import implementation.

## Review Checklist
- [ ] Storage layer changes are correct and efficient
- [ ] Index created properly on external_ref column
- [ ] Collision detection logic handles all edge cases
- [ ] Import flow correctly distinguishes external vs local issues
- [ ] No breaking changes to existing workflows
- [ ] Tests cover all scenarios (see bd-191)
- [ ] Documentation is clear and complete (see bd-183)
- [ ] Performance impact is acceptable
- [ ] Error messages are helpful
- [ ] Logging is appropriate for debugging

## Oracle Review
Use oracle tool to analyze implementation for correctness, edge cases, and potential issues.


Beads Library Integration

Migrate from custom SQLite implementation to using Beads as a library dependency. This eliminates ~3000 lines of duplicated code, reduces schema drift risk, and automatically inherits new Beads features.

**Key Benefits:**
- Remove 3000+ lines of duplicated SQLite code
- Eliminate schema drift between bd and vc CLIs
- Inherit Beads improvements automatically
- Stronger type safety with Beads error types
- Faster development velocity for new features
- Clean separation: Beads stays general-purpose, VC extends via wrapper

**Architecture Principle:**
Beads remains 100% standalone with NO VC dependencies. VC imports Beads (VC → Beads dependency) and wraps it with VC-specific storage methods. Both share the same database but maintain separate table namespaces.

**Current Pain Points:**
1. Code Duplication: Issue CRUD, dependency graphs, labels, status transitions all reimplemented
2. Schema Drift Risk: VC schema manually defined, could diverge from Beads
3. Lost Features: Can't leverage Beads query optimizer or advanced features without process spawning
4. Atomic Operations: Hand-rolled 100+ line transaction management
5. Maintenance Burden: Every Beads feature must be manually replicated

**Concrete Example:**
When Beads adds a new field (e.g., estimated_hours):
- Current: 4-6 hours of manual work (update types, 6+ SQL queries, migration, testing)
- With Library: 5 minutes (go get -u github.com/steveyegge/beads)

**Phased Approach:**
1. Phase 1: Add Beads dependency (non-breaking, feature flag)
2. Phase 2: Implement VCStorage wrapper (embeds beads.Storage)
3. Phase 3: Migration script for existing databases
4. Phase 4: Gradual cutover, deprecate SQLite code

**Related Analysis:**
See BEADS_INTEGRATION_ANALYSIS.md for detailed current state analysis and BEADS_LIBRARY_INTEGRATION_EPIC.md for full design document (both to be archived after issue creation).

**Estimated Effort:** 3-4 sprints
**Priority:** P2 (Medium-High - architectural improvement, high ROI)

Phase 1 (bd-137) complete! Beads can now be used as a Go library. VC can import github.com/steveyegge/beads and use beads.Storage directly instead of spawning CLI processes. No custom tables needed - VC uses pure Beads primitives.


Update documentation for external_ref import behavior

Document the new external_ref-based import matching behavior.

## Files to Update
- README.md: Add external_ref import section
- QUICKSTART.md: Add example of Jira/GitHub integration workflow
- AGENTS.md: Document external_ref import behavior for AI agents
- FAQ.md: Add Q&A about external system integration
- Add examples/ directory entry showing Jira/GitHub/Linear integration

## Key Points to Document
- How external_ref matching works
- That local issues are protected from external imports
- Example workflow: import → add local tasks → re-import updates
- ID conflict resolution behavior


Test description update

Updated with daemon running


Phase 2: Implement VCStorage Wrapper

Create VCStorage wrapper that embeds beads.Storage and adds VC-specific operations.

**Goal:** Build clean abstraction layer where VC extends Beads without modifying Beads library.

**Architecture:**
- VCStorage embeds beads.Storage (delegates core operations)
- VCStorage adds VC-specific methods (executor instances, events)
- Same database, separate table namespaces (Beads tables + VC tables)
- Zero changes to Beads library code

**Key Tasks:**
1. Create VCStorage struct that embeds beads.Storage
2. Implement VC-specific methods: CreateExecutorInstance(), GetStaleExecutors(), LogEvent(), UpdateExecutionState()
3. Create VC table schemas (executor_instances, issue_execution_state, agent_events)
4. Verify type compatibility between VC types.Issue and Beads Issue
5. Create MockVCStorage for testing
6. Write unit tests for VC-specific methods
7. Write integration tests (end-to-end with Beads)
8. Benchmark performance vs current SQLite
9. Verify NO changes needed to Beads library

**Acceptance Criteria:**
- VCStorage successfully wraps Beads storage (embedding works)
- VC-specific tables created and accessible via foreign keys to Beads tables
- VC-specific methods work (executor instances, events)
- Core operations delegate to Beads correctly
- Tests pass with >90% coverage
- Performance benchmark shows no regression
- Beads library remains unmodified and standalone

**Technical Details:**
- Use beadsStore.DB() to get underlying database connection
- Create VC tables with FOREIGN KEY references to Beads issues table
- Schema separation: Beads owns (issues, dependencies, labels), VC owns (executor_instances, agent_events)
- Testing: Embed MockBeadsStorage in MockVCStorage

**Dependencies:**
- Blocked by Phase 1 (need Beads library imported)

**Estimated Effort:** 1.5 sprints


Phase 3: Migration Path & Database Schema Alignment

Enable existing .beads/vc.db files to work with Beads library through automated migration.

**Goal:** Provide safe, tested migration path from SQLite implementation to Beads library.

**Key Tasks:**
1. Run compatibility tests against production databases
2. Identify schema differences (columns, indexes, constraints)
3. Document required migrations
4. Create migration CLI command: 'vc migrate --from sqlite --to beads'
5. Add dry-run mode for preview
6. Add backup/restore capability
7. Implement rollback mechanism
8. Add auto-detection of schema version on startup
9. Add auto-migrate with user prompt

**Acceptance Criteria:**
- Existing databases migrate successfully
- Data integrity preserved (zero data loss verified via checksums)
- Rollback works if migration fails
- Migration tested on real production VC databases
- Dry-run mode shows exactly what will change
- Backup created before migration
- Feature flag: VC_FORCE_SQLITE=true provides escape hatch

**Technical Details:**
- Compare current SQLite schema with Beads schema
- Handle version detection (read schema_version or detect from structure)
- Migration should be idempotent (safe to run multiple times)
- Backup strategy: Copy .beads/vc.db to .beads/vc.db.backup-<timestamp>
- Verify foreign key integrity after migration

**Safety Measures:**
- Require executor shutdown before migration (check for running executors)
- Atomic migration (BEGIN IMMEDIATE transaction)
- Comprehensive pre/post migration validation
- Clear error messages with recovery instructions

**Dependencies:**
- Blocked by Phase 2 (need VCStorage implementation)

**Estimated Effort:** 0.5 sprint


Phase 4: Gradual Cutover & Production Rollout

Replace SQLite implementation with Beads library in production and remove legacy code.

**Goal:** Complete transition to Beads library, deprecate and remove custom SQLite implementation.

**Key Tasks:**
1. Run VC executor with Beads library in CI
2. Dogfood: Use Beads library for VC's own development
3. Monitor for regressions and performance issues
4. Flip feature flag: VC_USE_BEADS_LIBRARY=true by default
5. Monitor production logs for errors
6. Collect user feedback
7. Add deprecation notice to CLAUDE.md
8. Provide migration guide for users
9. Remove legacy code: internal/storage/sqlite/sqlite.go (~1500 lines)
10. Remove migration framework: internal/storage/migrations/
11. Remove manual transaction management code
12. Update all documentation

**Acceptance Criteria:**
- Beads library enabled by default in production
- Zero production incidents related to migration
- Performance meets or exceeds SQLite implementation
- All tests passing with Beads library
- Legacy SQLite code removed
- Documentation updated
- Celebration documented 🎉

**Rollout Strategy:**
1. Week 1: Enable for CI/testing environments
2. Week 2: Dogfood on VC development
3. Week 3: Enable for 50% of production (canary)
4. Week 4: Enable for 100% of production
5. Week 5: Remove legacy code

**Monitoring:**
- Track error rates before/after cutover
- Monitor database query performance
- Track issue creation/update latency
- Monitor executor claim performance

**Rollback Plan:**
- Keep VC_FORCE_SQLITE=true escape hatch for 2 weeks post-cutover
- Keep legacy code for 1 sprint after cutover
- Document rollback procedure

**Success Metrics:**
- Zero data loss
- No performance regression (< 5% latency increase acceptable)
- Reduced maintenance burden (code LOC reduction)
- Positive developer feedback

**Dependencies:**
- Blocked by Phase 3 (need migration tooling)

**Estimated Effort:** 1 sprint


Example library-created issue

This issue was created programmatically using Beads as a library


Add lifecycle safety docs and tests for UnderlyingDB() method

The new UnderlyingDB() method exposes the raw *sql.DB connection for extensions like VC to create their own tables. While database/sql is concurrency-safe, there are lifecycle and misuse risks that need documentation and testing.

**What needs to be done:**

1. **Enhanced documentation** - Expand UnderlyingDB() comments to warn:
   - Callers MUST NOT call Close() on returned DB
   - Do NOT change pool/driver settings (SetMaxOpenConns, SetConnMaxIdleTime)
   - Do NOT modify SQLite PRAGMAs (WAL mode, journal, etc.)
   - Expect errors after Storage.Close() - use contexts
   - Keep write transactions short to avoid blocking core storage

2. **Add lifecycle tracking** - Implement closed flag:
   - Add atomic.Bool closed field to SQLiteStorage
   - Set flag in Close(), clear in New()
   - Optional: Add IsClosed() bool method

3. **Add safety tests** (run with -race):
   - TestUnderlyingDB_ConcurrentAccess - N goroutines using UnderlyingDB() during normal storage ops
   - TestUnderlyingDB_AfterClose - Verify operations fail cleanly after storage closed
   - TestUnderlyingDB_CreateExtensionTables - Create VC table with FK to issues, verify FK enforcement
   - TestUnderlyingDB_LongTxDoesNotCorrupt - Ensure long read tx doesn't block writes indefinitely

**Why this matters:**
VC will use this to create tables in the same database. Need to ensure production-ready safety without over-engineering.

**Estimated effort:** S+S+S = M total (1-3h)


MCP close tool method signature error - takes 1 positional argument but 2 were given

The close approval routing fix in beads-mcp v0.11.0 works correctly and successfully routes update(status="closed") calls to close() tool. However, the close() tool has a Python method signature bug that prevents execution.

Impact: All MCP-based close operations are broken. Workaround: Use bd CLI directly.

Error: BdDaemonClient.close() takes 1 positional argument but 2 were given

Root cause: BdDaemonClient.close() only accepts self, but MCP tool passes issue_id and reason.

Additional issue: CLI close has FOREIGN KEY constraint error when recording reason parameter.

See GitHub issue #107 for full details.


Update EXTENDING.md with UnderlyingDB() usage and best practices

EXTENDING.md currently shows how to use direct sql.Open() to access the database, but doesn't mention the new UnderlyingDB() method that's the recommended way for extensions.

**Update needed:**
1. Add section showing UnderlyingDB() usage:
   ```go
   store, err := beads.NewSQLiteStorage(dbPath)
   db := store.UnderlyingDB()
   // Create extension tables using db
   ```

2. Document when to use UnderlyingDB() vs direct sql.Open():
   - Use UnderlyingDB() when you want to share the storage connection
   - Use sql.Open() when you need independent connection management

3. Add safety warnings (cross-reference from UnderlyingDB() docs):
   - Don't close the DB
   - Don't modify pool settings
   - Keep transactions short

4. Update the VC example to show UnderlyingDB() pattern

5. Explain beads.Storage.UnderlyingDB() in the API section


Write comprehensive tests for external_ref import scenarios

Create test suite covering all external_ref import behaviors.

## Test Cases
1. Import with external_ref → creates new issue
2. Re-import same external_ref with changes → updates existing
3. Re-import same external_ref unchanged → idempotent (no update)
4. Import external issue with ID collision → auto-remaps external issue
5. Import without external_ref → uses current ID-based logic
6. Mixed batch: some with external_ref, some without
7. External_ref match + ID mismatch → updates by external_ref
8. Local issue (no external_ref) never matched by external import
9. Performance test with external_ref index

Add tests to:
- cmd/bd/import_shared_test.go
- internal/storage/sqlite/collision_test.go


Update import flow to handle external_ref updates

Modify importIssuesCore in import_shared.go to handle external_ref-based updates.

## Changes
- Phase 4: When processing issues, check collision result for external_ref matches
- Track external_ref updates separately from ID-based updates
- Ensure local issues (no external_ref) are never overwritten by imports
- Handle ID conflicts when external issue ID matches local issue ID (auto-remap)
- Update result reporting to show external_ref updates vs ID updates


Remove unused issueMap in scoreCollisions

scoreCollisions() creates issueMap and populates it (lines 135-138) but never uses it. Either remove it or add a TODO comment explaining future use. Located in collision.go:135-138. Cosmetic cleanup.


Issue counter gets out of sync with actual issues

The issue counter in issue_counters table frequently desyncs from actual max issue ID, causing:
- Import from JSONL leaves counter at old high value
- Test pollution increments counter but cleanup doesn't decrement it
- Delete issues doesn't update counter
- Only fix is 'rm -rf .beads' which is destructive

Examples from today's session:
- Had 48 issues but counter at 7714 after test pollution
- Import from git didn't reset counter
- Next new issue would be bd-7715 instead of bd-245

Proposed fixes:
1. Auto-recalculate counter from max(issue_id) on import
2. Add 'bd fix-counter' command
3. Make counter lazy (always compute from DB, don't store)
4. Import should reset counter to match imported data

Related: bd-361 (test isolation), bd-380 (init timestamp bug)


Add findByExternalRef query and database index

Implement storage layer support for looking up issues by external_ref field.

## Tasks
- Add findByExternalRef(ctx, externalRef) method to SQLiteStorage
- Add database index: CREATE INDEX IF NOT EXISTS idx_issues_external_ref ON issues(external_ref)
- Handle nil/empty external_ref cases properly
- Add unit tests for the new query method


CI still failing despite local lint passing - config v2 issue

Despite fixing all lint issues locally (golangci-lint reports 0 issues), CI continues to fail for dependabot PRs and test/lint checks.

**Local environment:**
- golangci-lint v2.5.0
- .golangci.yml version: "2" (v2 config format)
- Result: 0 issues ✅

**CI environment:**
- golangci/golangci-lint-action@v8
- golangci-lint version: v2.5.0
- Result: FAILING ❌

**Root cause suspects:**

1. **Config v2 compatibility**: The golangci-lint-action@v8 may not properly support config version "2" format, causing exclusion rules to be ignored

2. **Environment differences**:
   - CI runs on Ubuntu (linux/amd64)
   - Local runs on macOS (darwin/arm64)
   - Pattern matching in exclusions might behave differently

3. **Default linters**: CI might enable different default linters that aren't covered by our exclusions

4. **Config parsing**: The action might not properly read .golangci.yml or apply settings correctly

**Evidence:**
- Commits 963181d and c2c7eda fixed issues locally
- git push succeeded
- But CI still showing failures on dependabot PRs

**Investigation steps:**

1. Access actual CI logs from failing runs to see exact error messages
2. Test exact CI command locally: `golangci-lint run --timeout=5m ./...`
3. Check if .golangci.yml is being loaded by adding debug output
4. Compare default linters between local and CI: `golangci-lint linters`
5. Test config v1 format vs v2 format

**Potential fixes:**

A. **Switch to config v1**: Revert .golangci.yml to version 1 format (older but more stable)

B. **Explicit linter control**: Use `disable-all: true` and only enable specific linters

C. **Update action version**: Try newer golangci-lint-action or pin different golangci-lint version

D. **Inline exclusions**: Move exclusions from YAML to nolint directives in code

E. **CI override**: Add `args: --disable=gocyclo,gosec` to CI workflow

- CI passes lint checks with 0 issues
- Dependabot PRs can merge successfully
- Local and CI environments report same lint results
- Config is maintainable and exclusions work reliably

Fixed: added 'version: "2"' to .golangci.yml and upgraded golangci-lint-action from v6 to v8 with version: latest


Investigate and upgrade to modernc.org/sqlite 1.39.1+

We had to pin modernc.org/sqlite to v1.38.2 due to a FOREIGN KEY constraint regression in v1.39.1 (SQLite 3.50.4).

**Issue:** bd-200, GH #144

**Symptom:** CloseIssue fails with "FOREIGN KEY constraint failed (787)" when called via MCP/daemon, but works fine via CLI.

**Root Cause:** Unknown - likely stricter FK enforcement in SQLite 3.50.4 or modernc.org wrapper changes.

**Workaround:** Pinned to v1.38.2 (SQLite 3.49.x)

**TODO:**
1. Monitor modernc.org/sqlite releases for fixes
2. Check SQLite 3.50.5+ changelogs for FK-related fixes
3. Investigate why daemon mode fails but CLI succeeds (connection reuse? transaction isolation?)
4. Consider filing upstream issue with reproducible test case
5. Upgrade when safe


Test FK regression fix


Git worktree support - ensure --no-daemon works correctly

Users working with git worktrees report that beads MCP commits to the wrong branch (main instead of worktree branch).

**Root cause:** Git worktrees share the same .beads directory, so the daemon/MCP server doesn't know which branch the worktree has checked out.

**Current state:**
- Daemon mode: Cannot work properly with worktrees (fundamental limitation - shared .beads, unknown branch)
- CLI with --no-daemon: Should work but needs verification

**Action items:**
1. Test and verify --no-daemon works correctly in worktrees
2. Document the limitation in README/AGENTS.md
3. Add clear error/warning when using daemon with worktrees
4. Consider if MCP server can detect worktree usage and auto-disable daemon

**Workaround for users:**
Use --no-daemon flag when working in worktrees, or set BEADS_AUTO_DAEMON=false

**Related:** GH issue #55

**Implementation completed with oracle review improvements:**

1. **Robust detection** - Changed from substring check to canonical git-dir vs git-common-dir comparison
2. **Correct warning gate** - Now only checks isGitWorktree() and only called when daemon is actually connected
3. **Complete coverage** - Added warning to `bd daemon` command start path
4. **Better UX** - Shows shared database path and clarifies BEADS_AUTO_START_DAEMON behavior
5. **Improved tests** - Validates canonical detection method and path truncation

**Files changed:**
- cmd/bd/worktree.go - Improved detection and warning
- cmd/bd/worktree_test.go - Better test coverage
- cmd/bd/main.go - Warning after daemon connection (2 places)
- cmd/bd/daemon.go - Warning when starting daemon
- README.md & AGENTS.md - Documented limitations and solutions


Improve error handling in dependency removal during remapping

In updateDependencyReferences(), RemoveDependency errors are caught and ignored with continue (line 392). Comment says 'if dependency doesn't exist' but this catches ALL errors including real failures. Should check error type with errors.Is(err, ErrDependencyNotFound) and only ignore not-found errors, returning other errors properly.


Fix pre-existing MCP test failures - show/update return arrays not dicts

9 tests fail in beads-mcp because bd CLI commands return arrays but MCP client expects dicts:

Failing tests:
- test_create_and_show_issue: show returns array, expects dict
- test_update_issue: update returns array, expects dict
- test_add_dependency: show returns array, expects dict
- test_invalid_issue_id: show returns empty dict instead of error
- test_dependency_types: show returns array, expects dict
- test_show_issue_tool: show returns array, expects dict
- test_update_issue_tool: update returns array, expects dict
- test_update_partial_fields: update returns array, expects dict
- test_client_lazy_initialization: BdClient import issue

Root cause: bd CLI commands like 'bd show' and 'bd update' output JSON arrays, but BdCliClient.show() and BdCliClient.update() expect single dict objects.

Example:
```bash
bd show test-1 --json
[{"id":"test-1",...}]  # Array, not dict
```

Fix needed: Update bd_client.py to handle array responses and extract first element, or change CLI to return single object for single-ID operations.


Investigate GH#144: FOREIGN KEY regression in v0.16.0

v0.16.0 introduced FOREIGN KEY constraint error when closing issues via MCP (daemon mode). CLI works fine.

**Key Finding**: Only change between v0.15.0 and v0.16.0 that could affect FK behavior is:
- modernc.org/sqlite bumped from 1.38.2 to 1.39.1 (commit fbe63bf)

**Evidence**:
- User reports v0.15.0 worked perfectly (Oct 24, 04:35 UTC)
- v0.16.0 fails with 'constraint failed: FOREIGN KEY constraint failed (787)'
- Error occurs in both close() tool and update(status=closed) smart routing
- CLI 'bd close' works fine, only MCP/daemon fails
- No changes to CloseIssue() implementation between versions
- No changes to schema or RPC server handlers

**Next Steps**:
1. Test downgrading sqlite to 1.38.2
2. Check modernc.org/sqlite changelog for FK-related changes
3. Reproduce locally with MCP
4. If sqlite is the culprit, either fix or pin version

**Related**: GH #144