Cache getMultiRepoJSONLPaths() result during sync to avoid redundant calls

From bd-xo6b code review: getMultiRepoJSONLPaths() is called 3x per sync cycle.

**Current behavior:**
daemon_sync.go calls getMultiRepoJSONLPaths() three times per sync:
- Line 505: Snapshot capture before pull
- Line 575: Merge/prune after pull
- Line 613: Base snapshot update after import

**Cost per call:**
- Config lookup (likely cached, but still overhead)
- Path construction: O(N) where N = number of repos
- String allocations: (N + 1) × filepath.Join() calls

**Total per sync:** 3N path constructions + 3 config lookups + 3 slice allocations

**Impact:**
- For N=3 repos: Negligible (< 1ms)
- For N=10 repos: Still minimal
- For N=100+ repos: Wasteful

**Solution:**
Call once at sync start, reuse result:

```go
func createSyncFunc(...) func() {
    return func() {
        // ... existing setup ...

        // Call once at start
        multiRepoPaths := getMultiRepoJSONLPaths()

        // Snapshot capture
        if multiRepoPaths != nil {
            for _, path := range multiRepoPaths {
                if err := captureLeftSnapshot(path); err != nil { ... }
            }
        }

        // ... later ...

        // Merge/prune - reuse same paths
        if multiRepoPaths != nil {
            for _, path := range multiRepoPaths { ... }
        }

        // ... later ...

        // Base snapshot update - reuse same paths
        if multiRepoPaths != nil {
            for _, path := range multiRepoPaths { ... }
        }
    }
}
```

**Files:**
- cmd/bd/daemon_sync.go:449-636 (createSyncFunc)

**Note:** This is a performance optimization, not a correctness fix. Low priority unless multi-repo usage scales significantly.


Integration Layer Implementation

Build the adapter layer that makes Agent Mail optional and non-intrusive.

Progress: bd-m9th (Python adapter library) completed with full test coverage. Next: bd-fzbg (update python-agent example with Agent Mail integration).


Standardize daemon detection: use tryDaemonLock probe before RPC

Before attempting RPC connection, call tryDaemonLock() to check if lock is held:
- If lock NOT held: skip RPC attempt (no daemon running)
- If lock IS held: proceed with RPC + short timeout

This is extremely cheap and eliminates unnecessary connection attempts.

Apply across all client entry points that probe for daemon.


Add performance benchmarks for multi-repo hydration

The contributor-workflow-analysis.md asserts sub-second queries (line 702) and describes smart caching via file mtime tracking (Decision #4, lines 584-618), but doesn't provide concrete performance benchmarks.

VC's requirement (from VC feedback section):
- Executor polls GetReadyWork() every 5-10 seconds
- Queries must be sub-second (ideally <100ms)
- Smart caching must avoid re-parsing JSONLs on every query

Suggested performance targets to validate:
- File stat overhead: <1ms per repo
- Hydration (when needed): <500ms for typical JSONL (<25k)
- Query (from cache): <10ms
- Total GetReadyWork(): <100ms (VC's requirement)

Also test at scale:
- N=1 repo (baseline)
- N=3 repos (typical)
- N=10 repos (edge case)

These benchmarks are critical for library consumers like VC that run automated polling loops.

- Performance benchmark suite created for multi-repo hydration
- Benchmarks cover file stat, hydration, and query times
- Tests at N=1, N=3, N=10 repo scales
- Results documented in contributor-workflow-analysis.md
- Performance targets met or issues filed for optimization


MCP server: "No workspace set" and "chunk longer than limit" errors

Two related errors reported in beads-mcp v0.21:

**Error 1: "No workspace set" after successful set_context**
```
✓ Set beads context
✗ list
  Error calling tool 'list': No workspace set. Either provide workspace_root
  parameter or call set_context() first.
```

Hypothesis: Environment variable persistence issue between MCP tool calls, or ContextVar not being set correctly by @with_workspace decorator.

**Error 2: "Separator is found, but chunk is longer than limit"**
```
✗ list
  Error calling tool 'list': Separator is found, but chunk is longer than limit
```

Hypothesis: MCP protocol output size limit exceeded. Large issue databases may produce JSON output that exceeds MCP stdio buffer limits.

Platform: Fedora 43, using copilot-cli with Sonnet 4.5

Workaround: CLI works fine (`bd list --status open --json`)

## Fixes Implemented

**Issue 1: "No workspace set" after successful set_context** ✅ FIXED

Root cause: os.environ doesn't persist across MCP tool calls. When set_context() set BEADS_WORKING_DIR in os.environ, that change was lost on the next tool call.

Solution:
- Added module-level _workspace_context dict for persistent storage (server.py:51)
- Modified set_context() to store in both persistent dict and os.environ (server.py:265-287)
- Modified with_workspace() decorator to check persistent context first (server.py:129-133)
- Updated where_am_i() to check persistent context (server.py:302-330)

**Issue 2: "chunk longer than limit"** ✅ FIXED

Root cause: MCP stdio protocol has buffer limits. Large issue lists with full dependencies/dependents exceed this.

Solution:
- Reduced default list limit from 50 to 20 (server.py:356, models.py:122)
- Reduced max list limit from 1000 to 100 (models.py:122)
- Strip dependencies/dependents from list() and ready() responses (server.py:343-350, 368-373)
- Full dependency details still available via show() command

## Testing

✅ Python syntax validated with py_compile
✅ Changes are backward compatible
✅ Persistent context falls back to os.environ for compatibility

Users should now be able to call set_context() once and have it persist across all subsequent tool calls. Large databases will no longer cause buffer overflow errors.


Add guidance for self-hosting projects

The contributor-workflow-analysis.md is optimized for OSS contributors making PRs to upstream projects. However, it doesn't address projects like VC that use beads for their own development (self-hosting).

Self-hosting projects differ from OSS contributors:
- No upstream/downstream distinction (they ARE the project)
- May run automated executors (not just humans)
- In bootstrap/early phase (stability matters)
- Single team/owner (not multiple contributors with permissions)

Guidance needed on:
- When self-hosting projects should stay single-repo (default, recommended)
- When they should adopt multi-repo (team planning, multi-phase dev)
- How automated executors should handle multi-repo (if at all)
- Special considerations for projects in bootstrap phase

Examples of self-hosting projects: VC (building itself with beads), internal tools, pet projects

- Section added: 'For Projects Using Beads for Self-Hosting'
- Clear guidance on when to stay single-repo vs adopt multi-repo
- Recommendations for automated executor behavior with multi-repo
- Bootstrap phase considerations documented


Review multi-repo deletion tracking implementation

Thoroughly review the multi-repo deletion tracking fix (bd-4oob):

Files changed:
- cmd/bd/deletion_tracking.go: Added getMultiRepoJSONLPaths() helper
- cmd/bd/daemon_sync.go: Updated snapshot capture/update logic for multi-repo
- cmd/bd/deletion_tracking_test.go: Added 2 new tests (287 lines)

Review focus areas:
1. Correctness: Does getMultiRepoJSONLPaths() handle all edge cases?
2. Performance: Calling getMultiRepoJSONLPaths() 3x per sync (snapshot capture, merge, base update) - should we cache?
3. Error handling: What if some repos fail snapshot operations but others succeed?
4. Race conditions: Multiple daemons in different repos?
5. Test coverage: Are TestMultiRepoDeletionTracking and TestMultiRepoSnapshotIsolation sufficient?
6. Path handling: Absolute vs relative paths, tilde expansion

This is fresh code - needs careful review before considering deletion tracking production-ready.

Code review completed. Overall assessment: Core deletion tracking logic is sound, but error handling and path handling issues make this not yet production-ready for multi-repo scenarios.

Key findings:

CRITICAL ISSUES (Priority 1):
1. Inconsistent error handling in daemon_sync.go - snapshot/merge fail hard but base update warns. Can leave DB in inconsistent state with no rollback. See bd-sjmr.
2. No path normalization in getMultiRepoJSONLPaths() - tilde expansion, relative paths, duplicates not handled. See bd-iye7.

SHOULD FIX (Priority 2):
3. Missing test coverage for edge cases - empty paths, duplicates, partial failures. See bd-kdoh.
4. Performance - getMultiRepoJSONLPaths() called 3x per sync (minor issue). See bd-we4p.

WHAT WORKS WELL:
- Atomic file operations with PID-based temp files
- Good snapshot isolation between repos
- Race condition protection via exclusive locks
- Solid test coverage for happy path scenarios

VERDICT: Address bd-iye7 and bd-sjmr before considering deletion tracking production-ready for multi-repo mode.

Detailed review notes available in conversation history.


Write Agent Mail integration guide

Comprehensive guide for setting up and using Agent Mail with Beads.

Acceptance Criteria:
- Installation instructions
- Configuration (environment variables)
- Architecture diagram
- Benefits and tradeoffs
- When to use vs not use
- Troubleshooting section
- Migration from git-only mode

File: docs/AGENT_MAIL.md

Sections:
- Quick start
- How it works
- Integration points
- Graceful degradation
- Multi-machine deployment
- FAQ


CLI tests (cli_fast_test.go) are slow and should be integration tests

The TestCLI_* tests in cmd/bd/cli_fast_test.go are taking 4-5 seconds each (40+ seconds total), making them the slowest part of the fast test suite.

Current timings:
- TestCLI_Import: 4.73s
- TestCLI_Blocked: 4.33s
- TestCLI_DepTree: 4.15s
- TestCLI_Close: 3.59s
- TestCLI_DepAdd: 3.50s
- etc.

These tests compile the bd binary once in init(), but then execute it multiple times per test with filesystem operations. Despite being named "fast", they're actually end-to-end CLI integration tests.

Options:
1. Tag with //go:build integration (move to integration suite)
2. Optimize: Use in-memory databases, reduce exec calls, better parallelization
3. Keep as-is but understand they're the baseline for "fast" tests

Total test suite currently: 13.8s (cmd/bd alone is 12.8s, and most of that is these CLI tests)

Fixed by reusing existing bd binary from repo root instead of rebuilding.

Before: 15+ minutes (rebuilding binary for every test package)
After: ~12 seconds (reuses pre-built binary)

The init() function now checks for ../../bd first before falling back to building. This means `go build && go test` is now fast.


bd config set succeeds but doesn't persist to config.toml

Commands like `bd config set daemon.auto_push true` return "Set daemon.auto_push = true" but the config file is never created and `bd info --json | jq '.config'` returns null.

**Steps to reproduce:**
1. Run `bd config set daemon.auto_push true`
2. See success message: "Set daemon.auto_push = true"
3. Check `cat .beads/config.toml` → file doesn't exist
4. Check `bd info --json | jq '.config'` → returns null

**Expected:**
- .beads/config.toml should be created with the setting
- bd info should show the config value

**Impact:**
Users can't enable auto-push/auto-commit via CLI as documented in AGENTS.md


Create deployment scripts for GCP

Automated provisioning scripts for GCP Compute Engine deployment.

Acceptance Criteria:
- Terraform/gcloud scripts
- Static IP allocation
- Firewall rules
- NGINX reverse proxy config
- TLS setup (Let's Encrypt)
- Systemd service file

File: deployment/agent-mail/gcp/


Prevent test pollution in production database

The bd-vxdr cleanup revealed test issues were created during manual testing in the production workspace (Nov 2-4, template feature development).

**Root cause:** Manual testing with `./bd create "Test issue"` pollutes the production .beads database.

**Prevention strategies:**
1. Use TEST_DB environment variable for manual testing
2. Add warning when creating issues with "Test" prefix
3. Improve developer docs about testing workflow
4. Consider adding `bd test-mode` command for isolated testing

**Implementation completed:**

1. ✅ Added warning when creating issues with "Test" prefix in production database
   - Shows yellow warning with ⚠ symbol
   - Suggests using BEADS_DB for isolated testing
   - Warning appears in create.go after title validation

2. ✅ Documented BEADS_DB testing workflow in AGENTS.md
   - Added "Testing Workflow" section in Development Guidelines
   - Includes manual testing examples with BEADS_DB
   - Includes automated testing examples with t.TempDir()
   - Clear warning about not polluting production database

3. ⚠ Decided against bd test-mode command
   - BEADS_DB already provides simple, flexible isolation
   - Additional command would add complexity without much benefit
   - Current approach follows Unix philosophy (env vars for config)

**Files modified:**
- cmd/bd/create.go - Added Test prefix warning
- AGENTS.md - Added Testing Workflow section

**Testing:**
- Verified warning appears when creating "Test" prefix issues
- Verified BEADS_DB isolation works correctly
- Built successfully with `go build`


bd export should verify JSONL line count matches database count

After export completes, bd should verify that the JSONL file line count matches the number of issues exported. This would catch silent failures where the export appears to succeed but doesn't actually write all issues.

Real-world scenario from VC project:
- Ran direct SQL DELETE to remove 240 issues
- Ran 'bd export -o .beads/issues.jsonl'
- No error shown, appeared to succeed
- But JSONL file was not updated (still had old line count)
- Later session found all 240 issues still in JSONL
- Had to repeat the cleanup

If export had verified line count, it would have immediately shown:
  Error: Export verification failed
    Expected: 276 issues
    JSONL file: 516 lines
    Mismatch indicates export failed to write all issues

This is especially important because:
1. JSONL is source of truth in git
2. Silent export failures cause data inconsistency
3. Users assume export succeeded if no error shown
4. The verification is cheap (just count lines)

Implementation:
- After writing JSONL, count lines in file
- Compare to len(exportedIDs)
- If mismatch, remove temp file and return error
- Show clear error message with both counts

In cmd/bd/export.go, after atomic rename (line ~301):

1. Count lines in final JSONL file:
   - Read file and count newlines
   - Or reuse countIssuesInJSONL() helper (already exists)

2. Compare to len(exportedIDs)

3. If mismatch:
   - Log error with both counts
   - Optionally: remove the bad JSONL file (or leave for debugging?)
   - Return error (exit 1)

4. Consider adding --skip-verify flag for edge cases

Edge cases:
- Partial line writes (corrupted file)
- File system issues
- Race conditions (another process modifying JSONL during export)

The countIssuesInJSONL() function already exists at line 20, can reuse it.

1. bd export verifies JSONL line count after write
2. Clear error shown if mismatch detected
3. Test case that simulates partial write failure
4. Does not affect export performance significantly (line counting is fast)


Test Agent Mail server failure scenarios

Verify graceful degradation across various failure modes.

Test Cases:
- Server never started
- Server crashes during operation
- Network partition (timeout)
- Server returns 500 error
- Invalid bearer token
- SQLite corruption

Acceptance Criteria:
- Agents continue working in all scenarios
- Clear log messages about degradation
- No crashes or data loss
- Beads JSONL remains consistent

File: tests/integration/test_mail_failures.py


Add tests for daemon vs non-daemon parity in list filters

After bd-o43 RPC integration, we need tests to verify daemon mode behaves identically to direct mode for all new filter flags.

Test coverage needed:
- Pattern matching: --title-contains, --desc-contains, --notes-contains
- Date ranges: all 6 date filter flags (created/updated/closed after/before)
- Empty/null checks: --empty-description, --no-assignee, --no-labels
- Priority ranges: --priority-min, --priority-max
- Status normalization: --status all vs no status flag
- Date parsing: YYYY-MM-DD, RFC3339, and error cases
- Backward compat: deprecated --label flag still works

Oracle review findings (bd-o43):
- Date parsing should support multiple formats
- Status 'all' should be treated as unset
- NoLabels field was missing from RPC protocol
- Error messages should be clear and actionable

Test approach:
- Create RPC integration tests in internal/rpc/server_issues_epics_test.go
- Compare daemon client.List() vs direct store.SearchIssues() for same filters
- Verify error messages match between modes
- Test with real daemon instance, not just unit tests


Create multi-agent race condition test

Automated test that runs 2+ agents simultaneously to verify collision prevention.

Acceptance Criteria:
- Script spawns 2 agents in parallel
- Both try to claim same issue
- Only one succeeds (via reservation)
- Other agent skips to different work
- Verify in JSONL that no duplicate claims
- Test with Agent Mail enabled/disabled

File: tests/integration/test_agent_race.py

Success Metric: Zero duplicate claims with Agent Mail, collisions without it


Daemons don't exit when parent process dies, causing accumulation and race conditions

Multiple daemon processes accumulate over time because daemons don't automatically stop when their parent process (e.g., coding agent) is killed. This causes:

1. Race conditions: 8+ daemons watching same .beads/beads.db, each with own 30s debounce timer
2. Git conflicts: Multiple daemons racing to commit/push .beads/issues.jsonl
3. Resource waste: Orphaned daemons from sessions days/hours old still running

Example: User had 8 daemons from multiple sessions (12:37AM, 7:20PM, 7:22PM, 7:47PM, 9:19PM yesterday + 9:54AM, 10:55AM today).

Solutions to consider:
1. Track parent PID and exit when parent dies
2. Use single global daemon instead of per-session
3. Document manual cleanup: pkill -f "bd daemon"
4. Add daemon lifecycle management (auto-cleanup of stale daemons)

Implementation complete:

1. Added ParentPID field to DaemonLockInfo struct (stored in daemon.lock JSON)
2. Daemon now tracks parent PID via os.Getppid() at startup
3. Both event loops (polling and event-driven) check parent process every 10 seconds
4. Daemon gracefully exits if parent process dies (detected via isProcessRunning check)
5. Handles edge cases:
   - ParentPID=0: Older daemons without tracking (ignored)
   - ParentPID=1: Adopted by init means parent died (exits)
   - Otherwise checks if parent process is still running

The fix prevents daemon accumulation by ensuring orphaned daemons automatically exit within 10 seconds of parent death.


Test dependency child