Investigate performance issues in VS Code Copilot (Windows)

Beads unusable in Windows 11 VS Code Copilot chat with Sonnet 4.5.
Summary event happens every 3-4 turns, taking 3 minutes.
Copilot summarizes after ~125k tokens despite model supporting 1M.
Large context size of beads might be triggering aggressive summarization.
Need workaround or optimization for context size.


Add Claude setup tip with dynamic priority

Add a predefined tip that suggests running `bd setup claude` when Claude Code is detected but not configured. This tip should have higher priority (shown more frequently) until the setup is complete.


Workaround for FastMCP outputSchema bug in Claude Code

The beads MCP server (v0.23.1) successfully connects to Claude Code, but all tools fail to load with a schema validation error due to a bug in FastMCP 2.13.1.

Error: "Invalid literal value, expected \"object\"" in outputSchema.

Root Cause: FastMCP generates outputSchema with $ref at root level without "type": "object" for self-referential models (Issue).

Workaround: Use slash commands (/beads:ready) or wait for FastMCP fix.


Create backup and restore procedures

Disaster recovery procedures for Agent Mail data.

Acceptance Criteria:
- Automated daily snapshots (GCP persistent disk)
- SQLite backup script
- Git repository backup
- Restore procedure documentation
- Test restore from backup

File: deployment/agent-mail/backup.sh


Improve timestamp comparison readability in import

The timestamp comparison logic uses double-negative which can be confusing:

Current code:
if !incoming.UpdatedAt.After(existing.UpdatedAt) {
    // skip update
}

More readable:
if incoming.UpdatedAt.After(existing.UpdatedAt) {
    // perform update
} else {
    // skip (local is newer)
}

This is a minor refactor for code clarity.

Related: bd-1022
Files: internal/importer/importer.go:411, 488


Refactor path canonicalization into helper function

The path canonicalization logic (filepath.Abs + EvalSymlinks) is duplicated in 3 places:
- beads.go:131-137 (BEADS_DIR handling)
- cmd/bd/main.go:446-451 (--no-db cleanup)
- cmd/bd/nodb.go:26-31 (--no-db initialization)

Refactoring suggestion:
Extract to a helper function like:
  func canonicalizePath(path string) string

This would:
- Reduce code duplication
- Make the logic easier to maintain
- Ensure consistent behavior across all path handling

Related to bd-e16b implementation.


Create Docker image for Agent Mail

Containerize Agent Mail server for easy deployment.

Acceptance Criteria:
- Dockerfile with Python 3.14
- Health check endpoint
- Volume mount for storage
- Environment variable configuration
- Multi-arch builds (amd64, arm64)

File: deployment/agent-mail/Dockerfile


Research hooks/skills approach for enforcing issue descriptions

## Solution: Hooks/Skills (mentioned in discussion)

Research the hooks/skills system mentioned by riordanpawley in discussion #366:
https://www.reddit.com/r/ClaudeAI/s/wrn2tjkMHX

## Investigation Tasks

1. Review the Reddit post to understand the approach
2. Determine if beads hooks can enforce validation
3. Check if Claude Code skills/hooks can intercept MCP calls
4. Assess feasibility and user burden

## Notes

From discussion #366:
> I'm using a skills/hooks system to get Claude to do that kind of thing right similar to https://www.reddit.com/r/ClaudeAI/s/wrn2tjkMHX

This might be a client-side solution rather than server-side.

## Deliverable

Document findings in issue notes, with recommendation on whether to pursue this approach.


Implement auto-merge functionality in duplicates command

The duplicates.go file has a TODO at line 95 to implement the performMerge function for automatic duplicate merging. Currently it just prints a warning message. This would automate the merge process instead of just suggesting commands.


troubleshoot GH#278 daemon exits every few secs


Add tests for helper functions (GetDirtyIssueHash, GetAllDependencyRecords, export hashes)

Several utility functions have 0% coverage:
- GetDirtyIssueHash (dirty.go)
- GetAllDependencyRecords (dependencies.go)
- GetExportHash, SetExportHash, ClearAllExportHashes (hash.go)

These are lower priority but should have basic coverage.


Consider extracting error handling helpers

Evaluate creating FatalError() and WarnError() helpers as suggested in ERROR_HANDLING.md to reduce boilerplate and enforce consistency. Prototype in a few files first to validate the approach.


Audit remaining cmd/bd files for error handling consistency

Extend ERROR_HANDLING_AUDIT.md to cover: daemon_sync.go, update.go, list.go, show.go, close.go, reopen.go, dep.go, label.go, comments.go, delete.go, compact.go, config.go, validate.go and other high-usage command files


Document CLI + hooks as recommended approach over MCP

Update documentation to position CLI + bd prime hooks as the primary recommended approach over MCP server, explaining why minimizing context matters even with large context windows (compute cost, energy, environment, latency).


Document error handling audit findings

Comprehensive audit document showing all error handling inconsistencies found across cmd/bd/*.go files, with specific examples and recommended refactorings


Proposal: Custom status states via config

Proposal to add 'custom status states' via `bd config`.
Users could define an optional issue status enum (e.g., awaiting_review, review_in_progress) in the config.
This would enable multi-step pipelines to process issues where each step correlates to a specific status.

Examples:
- awaiting_verification
- awaiting_docs
- awaiting_testing


Refactor: Move staleness check inside daemon branch

## Problem

Currently ensureDatabaseFresh() is called before the daemon mode check, but it checks daemonClient != nil internally and returns early. This is redundant.

**Location:** All read commands (list.go:196, show.go:27, ready.go:102, status.go:80, etc.)

## Current Pattern

Call happens before daemon check, function checks daemonClient internally.

## Better Pattern

Move staleness check to direct mode branch only, after daemon check.

## Impact
Low - minor performance improvement (avoids one function call per command in daemon mode)

## Effort
Medium - requires refactoring 8 command files

## Priority
Low - can defer to future cleanup PR


Update AGENTS.md with event-driven mode

Document BEADS_DAEMON_MODE env var. Explain opt-in during Phase 1. Add troubleshooting for watcher failures.


Import reports misleading '0 created, 0 updated' when actually importing all issues

When running 'bd import' on a fresh database (no existing issues), the command reports 'Import complete: 0 created, 0 updated' even though it successfully imported all issues from the JSONL file.

**Steps to reproduce:**
1. Delete .beads/beads.db
2. Run: bd import .beads/issues.jsonl
3. Observe output: 'Import complete: 0 created, 0 updated'
4. Run: bd list
5. Confirm: All issues are actually present in the database

**Expected behavior:**
Report the actual number of issues imported, e.g., 'Import complete: 523 created, 0 updated'

**Actual behavior:**
'Import complete: 0 created, 0 updated' (misleading - makes user think import failed)

**Impact:**
- Users think import failed when it succeeded
- Confusing during database sync operations (e.g., after git pull)
- Makes debugging harder (can't tell if import actually worked)

**Context:**
Discovered during VC session when syncing database after git pull. The misleading message caused confusion about whether the database was properly synced with the canonical JSONL file.


Add --start flag to bd daemon, show help with no args

Currently `bd daemon` with no args immediately starts the daemon. This is inconsistent with other daemon management commands like `--stop`, `--status`, etc.

## Proposed Changes

1. Add `--start` flag to explicitly start daemon
2. With no args or flags, print help text showing available options
3. Maintain backward compatibility where feasible

## Current Consumers

- **Auto-start logic** (`daemon_autostart.go:106, 270`): Calls `bd daemon` programmatically - needs update
- **MCP docs** (SETUP_DAEMON.md:111): Already incorrectly shows `bd daemon start` - will be fixed by this change
- **Python daemon client**: Suggests `bd daemon` in error messages - needs doc update

## Implementation Notes

- Default behavior (no args/flags) should show help
- `--start` should start daemon (current no-args behavior)
- Auto-start code needs to pass `--start` flag explicitly
- Update all documentation to use `bd daemon --start`
- Ensure backward compat doesn't break existing workflows


Add .beads/README.md during 'bd init' for project documentation and promotion

When 'bd init' is run, automatically generate a .beads/README.md file that:

1. Briefly explains what Beads is (AI-native issue tracking that lives in your repo)
2. Links to the main repository: https://github.com/steveyegge/beads
3. Provides a quick reference of essential commands:
   - bd create: Create new issues
   - bd list: View all issues
   - bd update: Modify issue status/details
   - bd show: View issue details
   - bd sync: Sync with git remote
4. Highlights key benefits for AI coding agents and developers
5. Encourages developers to try it out

The README should be enthusiastic and compelling to get open source contributors excited about using Beads for their AI-assisted development workflows.


Implement health checks in daemon event loop

Add health checks to checkDaemonHealth() function in daemon_event_loop.go:170:
- Database integrity check
- Disk space check
- Memory usage check

Currently it's just a no-op placeholder.


Improve compact/daemon/merge documentation and UX

Multiple documentation and UX issues encountered:
1. "bd compact --analyze" fails with misleading "requires SQLite storage" error when daemon is running. Needs --no-daemon or better error.
2. "bd merge" help text is outdated (refers to 3-way merge instead of issue merging).
3. Daemon mode purpose isn't clear to local-only users.
4. Compact/cleanup commands are hard to discover.

Proposed fixes:
- Fix compact+daemon interaction or error message.
- Update "bd merge" help text.
- Add "when to use daemon" section to docs.
- Add maintenance section to quickstart.


Update docs for import bug fix

Update AGENTS.md, README.md, TROUBLESHOOTING.md with import.orphan_handling config documentation. Document resurrection behavior, tombstones, config modes. Add troubleshooting section for import failures with deleted parents.


Test issue 2


bd find-duplicates - AI-powered duplicate detection

Find semantically duplicate issues.

Approaches:
1. Mechanical: Exact title/description matching
2. Embeddings: Cosine similarity (cheap, scalable)
3. AI: LLM-based semantic comparison (expensive, accurate)

Uses embeddings by default for >100 issues.

Files: cmd/bd/find_duplicates.go (new)


Implement `bd setup cursor` for Cursor IDE integration

Create a `bd setup cursor` command that integrates Beads workflow into Cursor IDE via .cursorrules file. Unlike Claude Code (which has hooks), Cursor uses a static rules file to provide context to its AI.


Add test coverage for internal/config package


Document inconsistent error handling strategy across codebase

**Scope:** Cross-cutting pattern across cmd/bd (create.go, init.go, sync.go, daemon_sync.go)

**Issue:** Three different error handling patterns used inconsistently:

**Pattern A: Exit immediately**
```go
if err := store.CreateIssue(...) {
    fmt.Fprintf(os.Stderr, "Error: %v\n", err)
    os.Exit(1)
}
```

**Pattern B: Warn and continue**
```go
if err := createConfigYaml(...) {
    fmt.Fprintf(os.Stderr, "Warning: %v\n", err)
    // Non-fatal - continue anyway
}
```

**Pattern C: Silent ignore**
```go
_ = store.Close()
```

**Impact:**
- Makes error handling non-deterministic
- Hard to test error paths
- Inconsistent user experience

**Task:**
1. Document when each pattern should be used
2. Add decision tree/flowchart to developer docs
3. Consider creating error handling helpers to enforce consistency
4. Audit codebase and refactor outliers


Critical: bd sync overwrites git-pulled JSONL instead of importing it

## Problem

When a user runs `git pull` (which updates .beads/beads.jsonl) followed by `bd sync`, the sync command fails to detect that the JSONL changed and exports the local database, **overwriting the pulled JSONL** instead of importing it first.

This causes cleaned-up issues (removed via `bd cleanup` in another repo) to be resurrected and pushed back to the remote.

## Root Cause

`hasJSONLChanged()` in cmd/bd/integrity.go incorrectly returns FALSE when it should return TRUE after git pull updates the JSONL file.

The function has two code paths:
1. **Fast-path (line 128-129)**: If mtime unchanged, return false
2. **Slow-path (line 135-151)**: Compute hash and compare

Suspected issue: The mtime-based fast-path may incorrectly return false if:
- Git preserves mtime when checking out files, OR
- The mtime check logic has a race condition, OR
- The stored last_import_mtime is stale/incorrect

## Reproduction

**Setup:**
1. Repo A: Has 686 issues (including 630 closed)
2. Run `bd cleanup -f` in Repo A (removes 630 closed issues → 56 issues remain)
3. Push to remote
4. Repo B: Still has 686 issues in database

**Trigger:**
1. In Repo B: `git pull` (JSONL now has 56 issues)
2. In Repo B: `bd sync`

**Expected behavior:**
- hasJSONLChanged() returns TRUE
- Auto-imports 56-issue JSONL
- Database updated to 56 issues
- Exports (no-op, DB and JSONL match)
- Pushes

**Actual behavior:**
- hasJSONLChanged() returns FALSE ❌
- Skips auto-import
- Exports 686 issues from DB to JSONL
- Commits and pushes the 686-issue JSONL
- **Undoes the cleanup from Repo A**

## Evidence

From actual session (2025-11-23):
- Commit 8bc8611 in cino/beads shows: `+640, -14 lines` after bd sync
- No "Auto-import complete" message in output
- Database had 686 issues, JSONL was pulled with 56 issues
- Sync exported DB → JSONL instead of importing JSONL → DB

## Impact

**Critical:** This breaks multi-device sync and causes:
- Lost work (cleanup operations undone)
- Data resurrection (deleted issues come back)
- Confusing merge conflicts
- User trust issues ("Why does sync keep bringing back deleted issues?")

## Proposed Solutions

### Option 1: Remove mtime fast-path (safest)
Always compute and compare hashes. Eliminates false negatives from mtime issues.

**Pros:** Guaranteed correctness
**Cons:** Slightly slower (hash computation on every sync)

### Option 2: Fix mtime comparison logic
Investigate why mtime check fails and fix it properly.

**Areas to check:**
- Does git preserve mtime on checkout? (may vary by git version/config)
- Is last_import_mtime updated correctly after all operations?
- Race condition between git pull and mtime check?

### Option 3: Add hash-based validation
Keep mtime fast-path but add hash validation as backup:
- If mtime unchanged, still spot-check hash occasionally (e.g., 10% of the time)
- Log warnings when mtime and hash disagree
- This would catch the bug while maintaining performance

### Option 4: Add `--force-import` flag to bd sync
Short-term workaround: Allow users to force import even if hasJSONLChanged() returns false.

## Workaround

Until fixed, after `git pull`:
```bash
bd import --force  # Force import the pulled JSONL
bd sync            # Then sync normally
```

Or manually run bd cleanup in affected repos.

## Files

- cmd/bd/integrity.go:101-152 (hasJSONLChanged function)
- cmd/bd/sync.go:130-143 (auto-import logic)
- cmd/bd/autoflush.go:698 (export updates last_import_hash)

## Related Issues

- bd-77gm: Import reports misleading counts
- bd-khnb: Content-based staleness detection (original implementation of hash checking)

## Fix Implemented (Option 1)

**Root Cause Confirmed:**
Git does NOT update mtime when checking out files. Testing confirmed that after `git checkout HEAD~1`, the file mtime remains unchanged even though content differs.

**Solution:**
Removed the mtime-based fast-path in `hasJSONLChanged()`. Now always computes content hash for comparison.

**Changes Made:**
1. **cmd/bd/integrity.go:107-116** - Removed mtime fast-path, always compute hash
2. **cmd/bd/sync.go:752** - Removed mtime storage after import
3. **cmd/bd/import.go:340** - Removed mtime storage after import
4. **cmd/bd/daemon_sync.go:280-301** - Removed mtime storage and updated comments
5. **cmd/bd/daemon_sync_test.go:479,607,628** - Removed mtime assertions from tests

**Performance Impact:**
Minimal. Hash computation takes ~10-50ms even for large databases, which is acceptable for sync operations.

**Testing:**
- All existing tests pass
- Test "mtime changed but content same - git operation scenario" verifies the fix
- Full test suite passes (cmd/bd and all internal packages)

**Verification:**
The fix ensures that after `git pull` updates the JSONL:
1. `hasJSONLChanged()` returns TRUE (content hash differs)
2. Auto-import runs and updates database
3. No data loss or resurrection of deleted issues


Add test coverage for internal/validation package


Code Review Sweep: thorough

Perform thorough code review sweep based on accumulated activity.

**AI Reasoning:**
Significant code volume added (150,273 lines) across multiple critical areas, including cmd/bd, internal/storage/sqlite, and internal/rpc. High file change count (616) indicates substantial refactoring or new functionality. The metrics suggest potential for subtle architectural or implementation issues that warrant review.

**Scope:** thorough
**Target Areas:** cmd/bd, internal/storage/sqlite, internal/rpc
**Estimated Files:** 12
**Estimated Cost:** $5

**Task:**
Review files for non-obvious issues that agents miss during focused work:
- Inefficiencies (algorithmic, resource usage)
- Subtle bugs (race conditions, off-by-one, copy-paste)
- Poor patterns (coupling, complexity, duplication)
- Missing best practices (error handling, docs, tests)
- Unnamed anti-patterns

File discovered issues with detailed reasoning and suggestions.


Add metadata distinction guidelines to ERROR_HANDLING.md

Update ERROR_HANDLING.md with clear guidelines distinguishing configuration metadata (Pattern A) from tracking metadata (Pattern B). Include examples from init.go and sync.go.


Improve Beads test coverage from 46% to 80%


Implement transaction retry logic for SQLITE_BUSY

BEGIN IMMEDIATE fails immediately on SQLITE_BUSY instead of retrying with exponential backoff.

Location: internal/storage/sqlite/sqlite.go:223-225

Problem:
- Under concurrent write load, BEGIN IMMEDIATE can fail with SQLITE_BUSY
- Current implementation fails immediately instead of retrying
- Results in spurious failures under normal concurrent usage

Solution: Implement exponential backoff retry:
- Retry up to N times (e.g., 5)
- Backoff: 10ms, 20ms, 40ms, 80ms, 160ms
- Check for context cancellation between retries
- Only retry on SQLITE_BUSY/database locked errors

Impact: Spurious failures under concurrent write load

Effort: 3 hours


Update documentation for bd prime and Claude integration

Update AGENTS.md, README.md, and QUICKSTART.md to document the new `bd prime` command, `bd setup claude` command, and tip system.


Clarify .beads/.gitattributes handling in Protected Branches docs

Protected Branches docs quick start leaves untracked `.beads` directory and `.gitattributes`.
Question: Are these changes meant to be checked into the protected branch?
Need to clarify if these should be ignored or committed, or if the instructions are missing a step.


Centralize error handling patterns in storage layer

80+ instances of inconsistent error handling across sqlite.go with mix of %w, %v, and no wrapping.

Location: internal/storage/sqlite/sqlite.go (throughout)

Problem:
- Some use fmt.Errorf("op failed: %w", err) - correct wrapping
- Some use fmt.Errorf("op failed: %v", err) - loses error chain
- Some return err directly - no context
- Hard to debug production issues
- Can't distinguish error types

Solution: Create internal/storage/sqlite/errors.go:
- Define sentinel errors (ErrNotFound, ErrInvalidID, etc.)
- Create wrapDBError(op string, err error) helper
- Convert sql.ErrNoRows to ErrNotFound
- Always wrap with operation context

Impact: Lost error context; inconsistent messages; hard to debug

Effort: 5-7 hours


Audit and enforce consistent error handling patterns across codebase

**Background:** Error handling patterns are now documented in docs/ERROR_HANDLING.md (bd-9lwr). We have three established patterns:
- Pattern A: Exit immediately (os.Exit) for fatal errors
- Pattern B: Warn and continue for optional operations
- Pattern C: Silent ignore for cleanup/best-effort

**Goal:** Systematically review all error handling in cmd/bd to ensure each instance uses the appropriate pattern according to the guidelines.

**Scope:**
- Review all files in cmd/bd/*.go
- Focus on files with high usage: create.go, init.go, sync.go, daemon_sync.go, export.go, import.go
- Identify outliers where similar operations use different patterns
- Refactor to use consistent patterns

**Acceptance Criteria:**
- Similar operations (e.g., all metadata updates) use the same pattern
- Critical operations always use Pattern A
- Auxiliary operations use Pattern B or C appropriately
- No mixed patterns for identical operation types

**References:**
- docs/ERROR_HANDLING.md - Guidelines and decision tree
- bd-9lwr - Documentation task
- bd-bwk2 - Related storage layer error handling work


Apply wrapDBError to remaining storage files


Fix failing integration tests in beads-mcp

The `beads-mcp` test suite has failures in `tests/test_bd_client_integration.py` (assertion error in `test_init_creates_beads_directory`) and errors in `tests/test_worktree_separate_dbs.py` (setup failures finding database). These need to be investigated and fixed to ensure a reliable CI baseline.


Apply wrapDBError to config.go


Latency test 1

Resetting stale in_progress status from old executor run (yesterday)


Add test coverage for internal/autoimport package


Rename last_import_hash metadata key to jsonl_content_hash

The metadata key 'last_import_hash' is misleading because it's updated on both import AND export (sync.go:614, import.go:320).

Better names:
- jsonl_content_hash (more accurate)
- last_sync_hash (clearer intent)

This is a breaking change requiring migration of existing metadata values.


bd validate - Comprehensive health check

Run all validation checks in one command.

Checks:
- Duplicates
- Orphaned dependencies
- Test pollution
- Git conflicts

Supports --fix-all for auto-repair.

Depends on bd-cbed9619.1, bd-0dcea000, bd-31aab707, bd-9826b69a.

Files: cmd/bd/validate.go (new)


Create deployment scripts for GCP

Automated provisioning scripts for GCP Compute Engine deployment.

Acceptance Criteria:
- Terraform/gcloud scripts
- Static IP allocation
- Firewall rules
- NGINX reverse proxy config
- TLS setup (Let's Encrypt)
- Systemd service file

File: deployment/agent-mail/gcp/


Performance Testing Framework

Add comprehensive performance testing for beads focusing on optimization guidance and validating 10K+ database scale. Uses standard Go tooling, follows existing patterns, minimal complexity.

Components:
- Benchmark suite for critical operations at 10K-20K scale
- Fixture generator for realistic test data (epic hierarchies, cross-links)
- User diagnostics via bd doctor --perf
- Always-on profiling integration

Goals:
- Identify bottlenecks for optimization work
- Validate performance at 10K+ issue scale
- Enable users to collect diagnostics for bug reports
- Support both SQLite and JSONL import paths


Add monitoring and alerting

Observability for production Agent Mail server.

Acceptance Criteria:
- Health check endpoint (/health)
- Prometheus metrics export
- Grafana dashboard
- Alerts for server downtime
- Alerts for high error rate
- Log aggregation config

File: deployment/agent-mail/monitoring/


Investigate TestRoutingIntegration failure

TestRoutingIntegration/maintainer_with_SSH_remote failed during pre-commit check with "expected role maintainer, got contributor".
This occurred while running `go test -short ./...` on darwin/arm64.
The failure appears unrelated to storage/sqlite changes.
Need to investigate if this is a flaky test or environmental issue.


False positive detection by Kaspersky Antivirus (Trojan)

Kaspersky Antivirus falsely detects beads (bd.exe v0.23.1) as a Trojan (PDM:Trojan.Win32.Generic) and removes it.
Event: Malicious object detected
Component: System Watcher
Object name: bd.exe

## Completed Actions

1. **Root cause analysis**: Documented that this is a known false positive with Go binaries
2. **Created comprehensive user documentation**: docs/ANTIVIRUS.md with:
   - Explanation of why Go binaries trigger false positives
   - Instructions for adding exclusions (Kaspersky, Windows Defender, etc.)
   - File integrity verification steps
   - False positive reporting procedures
   - FAQ section
3. **Updated TROUBLESHOOTING.md**: Added quick reference section with link to detailed guide

## Next Steps (Follow-up Issues)

The following actions require external processes or future releases:

1. **Submit false positive to Kaspersky**: Need to access bd.exe v0.23.1 Windows binary and submit to https://opentip.kaspersky.com/
2. **Code signing for Windows**: Requires purchasing certificate and configuring GoReleaser (medium-term)
3. **Apply for Kaspersky whitelist**: Long-term relationship building with vendor

## Current Status

Users affected by this issue now have:
- Clear documentation explaining the false positive
- Step-by-step instructions to work around it
- Ways to verify file integrity
- Process to report to their antivirus vendor

Build configuration already includes recommended optimizations (-s -w flags).


Create tip system infrastructure for contextual hints

Implement a tip/hint system that shows helpful contextual messages after successful commands. This is different from the existing error-path "Hint:" messages - tips appear on success paths to educate users about features they might not know about.


Clarify metadata handling in init.go (config vs tracking)

Review init.go:207-228 to ensure configuration metadata (issue_prefix, sync.branch) uses Pattern A consistently, while tracking metadata (bd_version, repo_id, etc.) uses Pattern B. Add comments explaining the distinction.


Support --reason flag in daemon mode for reopen command

The reopen.go command has a TODO at line 61 to add reason as a comment once RPC supports AddComment. Currently --reason flag is ignored in daemon mode with a warning.


Refactor AGENTS.md to use external references

Suggestion to use external references (e.g., "ALWAYS REFER TO ./beads/prompt.md") instead of including all instructions directly within AGENTS.md.
Reasons:
1. Agents can follow external references.
2. Prevents context pollution/stuffing in AGENTS.md as more tools append instructions.


Test parent issue


bd doctor --fix should auto-fix metadata.json jsonl_export mismatch

## Root Cause

The version tracking code (cmd/bd/version_tracking.go:33-38) auto-recreates metadata.json using DefaultConfig() if it doesn't exist. DefaultConfig() hardcodes jsonl_export to 'issues.jsonl', but many repos (including beads itself) use 'beads.jsonl'.

When metadata.json goes missing (git clean, merge conflict, rebase, etc.), the next bd command recreates it with the wrong filename, causing a mismatch with the actual JSONL file.

## Impact

This affects ANY user whose metadata.json gets deleted and who uses a non-default JSONL filename. Normal git operations can trigger this.

## Fix Requirements

1. bd doctor --fix should auto-detect and fix the mismatch (check which .jsonl file exists, update metadata.json)
2. Version tracking should detect actual JSONL file instead of using hardcoded default
3. DefaultConfig() should scan for existing .jsonl files before defaulting
4. Daemon should auto-detect config changes without restart

## Related Code

- internal/configfile/configfile.go:18-23 (DefaultConfig hardcodes issues.jsonl)
- cmd/bd/version_tracking.go:33-38 (auto-creates config with default)
- internal/beads/beads.go:245-246 (FindJSONLPath defaults to issues.jsonl)


Optimize CLI test suite - eliminate redundant git init calls

Current: Each of 13 CLI tests calls git init (31s total). Solution: Use single test binary built once in init(), skip git operations where possible, or use mock filesystem.


Fix autoimport tests for content-hash collision scoring

## Overview
Three autoimport tests are failing after bd-cbed9619.4 because they expect behavior based on the old reference-counting collision resolution, but the system now uses deterministic content-hash scoring.

## Failing Tests
1. `TestAutoImportMultipleCollisionsRemapped` - expects local versions preserved
2. `TestAutoImportAllCollisionsRemapped` - expects local versions preserved
3. `TestAutoImportCollisionRemapMultipleFields` - expects specific collision resolution behavior

## Root Cause
These tests were written when ScoreCollisions used reference counting to determine which version to keep. Now it uses content-hash comparison (introduced in commit 2e87329), which produces different but deterministic results.

## Example
Old behavior: Issue with more references would be kept
New behavior: Issue with lexicographically lower content hash is kept

## Solution
Update each test to:
1. Verify the new content-hash based behavior is correct
2. Check that the remapped issue (not necessarily local/remote) has the expected content
3. Ensure dependencies are preserved on the correct remapped issue

## Acceptance Criteria
- All three autoimport tests pass
- Tests verify content-hash determinism (same collision always resolves the same way)
- Tests check dependency preservation on remapped issues
- Test documentation explains content-hash scoring expectations

## Files to Modify
- `cmd/bd/autoimport_collision_test.go`

## Testing
Run: `go test ./cmd/bd -run "TestAutoImport.*Collision" -v`


cmd/bd test suite is absurdly slow - 279 tests taking 8+ minutes

# Problem

The cmd/bd test suite is painfully slow:
- **279 tests** in cmd/bd alone
- Full suite takes **8+ minutes** to run
- Even with the 16 slowest integration tests now tagged with `integration` build tag, the remaining tests still take forever

This makes the development loop unusable. We can't wait 8+ minutes every time we want to run tests.

# Root Cause Analysis

## 1. Sheer Volume
279 tests is too many for a single package. Even at 0.1s per test, that's 28 seconds minimum just for cmd/bd.

## 2. Each Test Creates Full Database + Temp Directories
Every test does heavy setup:
- Creates temp directory (`t.TempDir()` or `os.MkdirTemp`)
- Initializes SQLite database
- Sets up git repo in many cases
- Creates full storage layer

Example from the tests:
```go
func setupCLITestDB(t *testing.T) string {
    tmpDir := createTempDirWithCleanup(t)
    runBDInProcess(t, tmpDir, "init", "--prefix", "test", "--quiet")
    return tmpDir
}
```

This happens 279 times!

## 3. Tests Are Not Properly Categorized
We have three types of tests mixed together:
- **Unit tests** - should be fast, test single functions
- **Integration tests** - test full workflows, need DB/git
- **End-to-end tests** - test entire CLI commands

They're all lumped together in cmd/bd, all running every time.

# What We've Already Fixed

Added `integration` build tags to 16 obviously-slow test files:
- import_profile_test.go (performance benchmarking tests)
- export_mtime_test.go (tests with time.Sleep calls)
- cli_fast_test.go (full CLI integration tests)
- delete_test.go, import_uncommitted_test.go, sync_local_only_test.go (git integration)
- And 10 more in internal/ packages

These are now excluded from the default `go test ./...` run.

# Proposed Solutions

## Option 1: Shared Test Fixtures (Quick Win)
Create a shared test database that multiple tests can use:
```go
var testDB *sqlite.SQLiteStorage
var testDBOnce sync.Once

func getSharedTestDB(t *testing.T) storage.Storage {
    testDBOnce.Do(func() {
        // Create one DB for all tests
    })
    return testDB
}
```

**Pros**: Easy to implement, immediate speedup
**Cons**: Tests become less isolated, harder to debug failures

## Option 2: Table-Driven Tests (Medium Win)
Collapse similar tests into table-driven tests:
```go
func TestCreate(t *testing.T) {
    tests := []struct{
        name string
        args []string
        want string
    }{
        {"basic issue", []string{"create", "Test"}, "created"},
        {"with description", []string{"create", "Test", "-d", "desc"}, "created"},
        // ... 50 more cases
    }

    db := setupOnce(t) // Setup once, not 50 times
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            // test using shared db
        })
    }
}
```

**Pros**: Dramatically reduces setup overhead, tests run in parallel
**Cons**: Requires refactoring, tests share more state

## Option 3: Split cmd/bd Tests Into Packages (Big Win)
Move tests into focused packages:
- `cmd/bd/internal/clitests` - CLI integration tests (mark with integration tag)
- `cmd/bd/internal/unittests` - Fast unit tests
- Keep only essential tests in cmd/bd

**Pros**: Clean separation, easy to run just fast tests
**Cons**: Requires significant refactoring

## Option 4: Parallel Execution (Quick Win)
Add `t.Parallel()` to independent tests:
```go
func TestSomething(t *testing.T) {
    t.Parallel() // Run this test concurrently with others
    // ...
}
```

**Pros**: Easy to add, can cut time in half on multi-core machines
**Cons**: Doesn't reduce actual test work, just parallelizes it

## Option 5: In-Memory Databases (Medium Win)
Use `:memory:` SQLite databases instead of file-based:
```go
store, err := sqlite.New(ctx, ":memory:")
```

**Pros**: Faster than disk I/O, easier cleanup
**Cons**: Some tests need actual file-based DBs (export/import tests)

# Recommended Approach

**Short-term (this week)**:
1. Add `t.Parallel()` to all independent tests in cmd/bd
2. Use `:memory:` databases where possible
3. Create table-driven tests for similar test cases

**Medium-term (next sprint)**:
4. Split cmd/bd tests into focused packages
5. Mark more integration tests appropriately

**Long-term (backlog)**:
6. Consider shared test fixtures with proper isolation

# Current Status

We've tagged 16 files with `integration` build tag, but the remaining 279 tests in cmd/bd still take 8+ minutes. This issue tracks fixing the cmd/bd test performance specifically.

# Target

Get `go test ./...` (without `-short` or `-tags=integration`) down to **under 30 seconds**.

# THE REAL ROOT CAUSE (Updated Analysis)

After examining the actual test code, the problem is clear:

## Every Test Creates Its Own Database From Scratch

Look at `create_test.go`:
```go
func TestCreate_BasicIssue(t *testing.T) {
    tmpDir := t.TempDir()                          // ← Creates temp dir
    testDB := filepath.Join(tmpDir, ".beads", "beads.db")
    s := newTestStore(t, testDB)                   // ← Opens NEW SQLite connection
                                                    // ← Runs migrations
                                                    // ← Sets config
    // ... actual test (3 lines)
}

func TestCreate_WithDescription(t *testing.T) {
    tmpDir := t.TempDir()                          // ← Creates ANOTHER temp dir
    testDB := filepath.Join(tmpDir, ".beads", "beads.db")
    s := newTestStore(t, testDB)                   // ← Opens ANOTHER SQLite connection
    // ... actual test (3 lines)
}
```

**This happens 279 times!**

## These Tests Don't Need Isolation!

Most tests are just checking:
- "Can I create an issue with a title?"
- "Can I create an issue with a description?"
- "Can I add labels?"

They don't conflict with each other. They could all share ONE database!

## The Fix: Test Suites with Shared Setup

Instead of:
```go
func TestCreate_BasicIssue(t *testing.T) {
    s := newTestStore(t, t.TempDir()+"/db")  // ← Expensive!
    // test
}

func TestCreate_WithDesc(t *testing.T) {
    s := newTestStore(t, t.TempDir()+"/db")  // ← Expensive!
    // test
}
```

Do this:
```go
func TestCreate(t *testing.T) {
    // ONE setup for all subtests
    s := newTestStore(t, t.TempDir()+"/db")

    t.Run("basic_issue", func(t *testing.T) {
        t.Parallel()  // Can run concurrently - tests don't conflict
        // test using shared `s`
    })

    t.Run("with_description", func(t *testing.T) {
        t.Parallel()
        // test using shared `s`
    })

    // ... 50 more subtests, all using same DB
}
```

**Result**: 50 tests → 1 database setup instead of 50!

## Why This Works

SQLite is fine with concurrent reads and isolated transactions. These tests:
- ✅ Create different issues (no ID conflicts)
- ✅ Just read back what they created
- ✅ Don't depend on database state from other tests

They SHOULD share a database!

## Real Numbers

Current:
- 279 tests × (create dir + init SQLite + migrations) = **8 minutes**

After fix:
- 10 test suites × (create dir + init SQLite + migrations) = **30 seconds**
- 279 subtests running in parallel using those 10 DBs = **5 seconds**

**Total: ~35 seconds instead of 8 minutes!**

## Implementation Plan

1. **Group related tests** into suites (Create, List, Update, Delete, etc.)
2. **One setup per suite** instead of per test
3. **Use t.Run() for subtests** with t.Parallel()
4. **Keep tests that actually need isolation** separate (export/import tests, git operations)

This is way better than shuffling tests into folders!

## Progress Update (2025-11-21)

✅ **Completed**:
- Audited all 280 tests, created TEST_SUITE_AUDIT.md ([deleted:[deleted:[deleted:[deleted:[deleted:[deleted:[deleted:bd-c49]]]]]]])
- Refactored create_test.go to shared DB pattern ([deleted:[deleted:[deleted:[deleted:[deleted:[deleted:[deleted:bd-y6d]]]]]]])
- Proven the pattern works: 11 tests now run in 0.04s with 1 DB instead of 11

❌ **Current Reality**:
- Overall test suite: Still 8+ minutes (no meaningful change)
- Only 1 of 76 test files refactored
- Saved ~10 DB initializations out of 280

## Acceptance Criteria (REALISTIC)

This task is NOT complete until:
- [ ] All P1 files refactored (create ✅, dep, stale, comments, list, ready)
- [ ] Test suite runs in < 2 minutes
- [ ] Measured and verified actual speedup

## Next Steps

1. Refactor remaining 5 P1 files: dep_test.go, stale_test.go, comments_test.go, list_test.go, ready_test.go
2. Measure actual time improvement after each file
3. Continue with P2 files if needed to hit <2min target


Optimize Memory backend GetIssueByExternalRef with index

Currently GetIssueByExternalRef in Memory storage uses O(n) linear search through all issues.

Current code (memory.go:282-308):
for _, issue := range m.issues {
    if issue.ExternalRef != nil && *issue.ExternalRef == externalRef {
        return &issueCopy, nil
    }
}

Proposed optimization:
- Add externalRefToID map[string]string to MemoryStorage
- Maintain it in CreateIssue, UpdateIssue, DeleteIssue
- Achieve O(1) lookup like SQLite's index

Impact: Low (--no-db mode typically has smaller datasets)
Related: bd-1022


Add orphan detection migration

Create migration to detect orphaned children in existing databases. Query: SELECT id FROM issues WHERE id LIKE '%.%' AND substr(id, 1, instr(id || '.', '.') - 1) NOT IN (SELECT id FROM issues). Log results, let user decide action (delete orphans or convert to top-level).


Add test coverage for internal/git package


Apply wrapDBError to queries.go


CRITICAL: bd sync pushes stale DB state instead of trusting JSONL truth on pull

When a clone has stale DB (688 issues) and pulls fresh JSONL (62 issues), bd sync detects 1009.7% divergence but proceeds to re-export the stale DB and push it, overwriting the correct JSONL state.

Repro:
1. Clone has 688 issues in DB (628 closed)
2. git pull brings JSONL with 62 issues (cleanup applied)
3. bd sync warns about divergence but exports DB→JSONL
4. Pushes 688 issues back to remote, undoing cleanup

Expected: JSONL is source of truth after pull. DB should be wiped and reimported.
Actual: DB overwrites JSONL, pushing stale state upstream.

Impact: Breaks multi-clone coordination. Cleanup work gets undone.


Fix Homebrew warning about Ruby file location

Homebrew warning: Found Ruby file outside steveyegge/beads tap formula directory.
Warning points to: /opt/homebrew/Library/Taps/steveyegge/homebrew-beads/bd.rb
It should likely be inside a Formula/ directory or similar structure expected by Homebrew taps.


Add tests for error wrapping behavior


Import should continue on FOREIGN KEY constraint violations from deletions

# Problem

When importing JSONL after a merge that includes deletions, we may encounter FOREIGN KEY constraint violations if:
- Issue A was deleted in one branch
- Issue B (that depends on A) was modified in another branch
- The merge keeps the deletion of A and the modification of B
- Import tries to import B with a dependency/reference to deleted A

Currently import fails completely on such constraint violations, requiring manual intervention.

# Solution

Add IsForeignKeyConstraintError() helper similar to IsUniqueConstraintError()

Update import code to:
1. Detect FOREIGN KEY constraint violations
2. Log a warning with the issue ID and constraint
3. Continue importing remaining issues
4. Report summary of skipped issues at the end

# Implementation Notes

- Add to internal/storage/sqlite/util.go
- Pattern: strings.Contains(err.Error(), "FOREIGN KEY constraint failed")
- Update importer to handle these errors gracefully
- Keep track of skipped issues for summary reporting

## Implementation Complete

Added FOREIGN KEY constraint violation handling to the importer:

**Changes made:**

1. **internal/importer/importer.go**
   - Added SkippedDependencies field to Result struct
   - Updated importDependencies() to accept result parameter
   - Added FK constraint detection using sqlite.IsForeignKeyConstraintError()
   - Log warning for each skipped dependency
   - Track skipped dependencies in result

2. **cmd/bd/import_shared.go**
   - Added SkippedDependencies field to ImportResult struct
   - Updated result conversion to include skipped dependencies

3. **cmd/bd/import.go**
   - Added summary reporting for skipped dependencies
   - Displays warning with list of skipped dependencies and helpful context

**Behavior:**
- When a FOREIGN KEY constraint violation is encountered during dependency import:
  - A warning is logged: 'Warning: Skipping dependency due to missing reference: issue-a → issue-b (blocks)'
  - The dependency is tracked in result.SkippedDependencies
  - Import continues with remaining dependencies
  - Summary at end lists all skipped dependencies with context message

**Testing:**
- All existing importer tests pass
- Build succeeds
- Ready for real-world testing when FK constraint violations are encountered


Add explicit cache validation tests for blocked_issues_cache

The blocked_issues_cache optimization works correctly but lacks explicit tests that verify cache invalidation behavior.

## What to test

1. **Cache invalidation on dependency add**
   - Add a blocking dependency
   - Verify blocked issue appears in cache

2. **Cache invalidation on dependency remove**
   - Remove a blocking dependency
   - Verify blocked issue removed from cache

3. **Cache invalidation on status change**
   - Close a blocker issue
   - Verify dependent issue removed from cache
   - Reopen blocker
   - Verify dependent issue added back to cache

4. **Cache consistency across operations**
   - Multiple dependency changes in sequence
   - Verify cache stays consistent with actual blocking state

5. **Parent-child transitive blocking**
   - Create epic with child tasks
   - Add blocking dependency to epic
   - Verify all children appear in cache

## Implementation

Add tests to internal/storage/sqlite/blocked_cache_test.go (new file) with direct cache queries:

```go
func TestBlockedCacheInvalidation(t *testing.T) {
    // Verify cache updates correctly on dependency changes
}

func TestBlockedCacheConsistency(t *testing.T) {
    // Verify cache matches actual blocking state
}
```

## Why this matters

- Current tests verify behavior but not implementation
- Explicit cache tests catch cache-specific bugs
- Makes cache correctness visible in test output
- Easier debugging if cache gets out of sync

## Related

- [deleted:[deleted:[deleted:[deleted:[deleted:[deleted:bd-5qim]]]]]]: GetReadyWork performance optimization (implemented cache)
- internal/storage/sqlite/blocked_cache.go
- internal/storage/sqlite/ready_test.go (behavior tests)


Performance Testing Documentation

Create docs/performance-testing.md documenting the performance testing framework.

Sections:
1. Overview - What the framework does, goals
2. Running Benchmarks
   - make bench command
   - Running specific benchmarks
   - Interpreting output (ns/op, allocs/op)
3. Profiling and Analysis
   - Viewing CPU profiles with pprof
   - Reading flamegraphs
   - Memory profiling
   - Finding hotspots
4. User Diagnostics
   - bd doctor --perf usage
   - Sharing profiles with bug reports
   - Understanding the report output
5. Comparing Performance
   - Using benchstat for before/after comparisons
   - Detecting regressions
6. Tips for Optimization
   - Common patterns
   - When to profile vs benchmark

Style:
- Concise, practical examples
- Screenshots/examples of pprof output
- Clear command-line examples
- Focus on workflow, not theory


Fix mypy errors in beads-mcp

Running `mypy .` in `integrations/beads-mcp` reports 287 errors. These should be addressed to improve type safety and code quality.


Extract database search logic into helper function

The logic for finding a database in a beads directory is duplicated:
- FindDatabasePath() BEADS_DIR section (beads.go:141-169)
- findDatabaseInTree() (beads.go:248-280)

Both implement the same search order:
1. Check config.json first (single source of truth)
2. Fall back to canonical beads.db
3. Search for *.db files, filtering backups and vc.db

Refactoring suggestion:
Extract to a helper function like:
  func findDatabaseInBeadsDir(beadsDir string) string

Benefits:
- Single source of truth for database search logic
- Easier to maintain and update search order
- Reduces code duplication

Related to [deleted:[deleted:[deleted:[deleted:[deleted:[deleted:[deleted:[deleted:[deleted:bd-e16b]]]]]]]]] implementation.


Document blocked_issues_cache architecture and behavior

Add comprehensive documentation about the blocked_issues_cache optimization to help future maintainers understand the design.

## What to document

1. **Architecture overview**
   - Why cache exists (performance: 752ms -> 29ms)
   - When cache is used (GetReadyWork queries)
   - How cache is maintained (invalidation triggers)

2. **Cache invalidation rules**
   - Invalidated on: dependency add/remove (blocks/parent-child only)
   - Invalidated on: any status change
   - Invalidated on: issue close
   - NOT invalidated on: related/discovered-from dependencies

3. **Transaction safety**
   - All invalidations happen within transactions
   - Cache can use tx or direct db connection
   - Rebuild is atomic (DELETE + INSERT in same tx)

4. **Performance characteristics**
   - Full rebuild on every invalidation
   - Rebuild is fast (<50ms on 10K database)
   - Write complexity traded for read speed
   - Dependency changes are rare vs reads

5. **Edge cases**
   - Parent-child transitive blocking
   - Closed issues don't block
   - Foreign key CASCADE handles deletions

## Where to document

Add to these locations:

1. **blocked_cache.go** - Add detailed package comment:
```go
// Package blocked_cache implements the blocked_issues_cache optimization.
//
// Performance: GetReadyWork originally used recursive CTE (752ms on 10K db).
// With cache: Simple NOT EXISTS query (29ms on 10K db).
//
// Cache Maintenance:
// - Rebuild triggered on dependency changes (blocks/parent-child only)
// - Rebuild triggered on status changes
// - Full rebuild (DELETE + INSERT) is fast enough for all cases
// ...
```

2. **ARCHITECTURE.md** or **PERFORMANCE.md** (new file) - High-level overview

3. **ready.go** - Update comment at line 84-85 with more detail

## Related

- [deleted:[deleted:[deleted:[deleted:[deleted:[deleted:bd-5qim]]]]]]: GetReadyWork optimization (implemented)
- bd-13gm: Cache validation tests
- [deleted:[deleted:[deleted:[deleted:bd-obdc]]]]: Cache rebuild command


Add programmatic tip injection API

Allow tips to be programmatically injected at runtime based on detected conditions. This enables dynamic tips (not just pre-defined ones) to be shown with custom priority and frequency.