Implements automatic fix capability for bd doctor with user confirmation
and security hardening.
Features:
- Organizes fix implementations under doctor/fix/ directory structure
- Shows all fixable issues and prompts for confirmation (Y/n) before applying
- Provides clear output about what was fixed and any errors encountered
- Re-runs diagnostics after fixes to show updated state
- Each fix is idempotent and safe to run multiple times
Automatic fixes implemented:
- Git hooks (runs bd hooks install)
- Daemon health issues (runs bd daemons killall)
- DB-JSONL sync problems (runs bd sync --import-only)
- File permissions (fixes .beads/ and database permissions)
- Database version mismatches (runs bd migrate)
- Schema compatibility issues (runs bd migrate)
- Gitignore updates (writes canonical template)
Security improvements:
- Prevents command injection by using os.Executable() instead of PATH lookup
- Prevents path traversal attacks with workspace validation
- Fixes race conditions by using cmd.Dir instead of os.Chdir()
- Corrects file permission logic (proper bit masking)
- Validates all operations run in beads workspaces only
Files changed:
- cmd/bd/doctor.go: Enhanced applyFixes() with confirmation and better UX
- cmd/bd/doctor/gitignore.go: Fixed permissions (0600 → 0644)
- cmd/bd/doctor/fix/common.go: Security helpers (getBdBinary, validateBeadsWorkspace)
- cmd/bd/doctor/fix/hooks.go: Git hooks fix
- cmd/bd/doctor/fix/daemon.go: Daemon health fix
- cmd/bd/doctor/fix/sync.go: DB-JSONL sync fix
- cmd/bd/doctor/fix/permissions.go: File permissions fix
- cmd/bd/doctor/fix/migrate.go: Database migration fixes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Enhanced checkIDFormat to sample multiple issues instead of just one
- Added detectHashBasedIDs function with robust multi-heuristic detection:
* Checks for child_counters table (hash ID schema indicator)
* Detects letters in IDs (base36 encoding)
* Identifies leading zeros (common in hash IDs, rare in sequential)
* Analyzes variable length patterns (adaptive hash IDs)
* Checks for non-sequential numeric ordering
- Added comprehensive test coverage (16 new test cases)
- Fixes false positives for numeric-only hash IDs like 'pf-0088'
Closes#322
* feat: add performance testing framework foundation
Implements foundation for comprehensive performance testing and user
diagnostics for beads databases at 10K-20K scale.
Components added:
- Fixture generator (internal/testutil/fixtures/) for realistic test data
* LargeSQLite/XLargeSQLite: 10K/20K issues with epic hierarchies
* LargeFromJSONL/XLargeFromJSONL: test JSONL import path
* Realistic cross-linked dependencies, labels, assignees
* Reproducible with seeded RNG
- User diagnostics (bd doctor --perf) for field performance data
* Collects platform info (OS, arch, Go/SQLite versions)
* Measures key operation timings (ready, list, show, search)
* Generates CPU profiles for bug reports
* Clean separation in cmd/bd/doctor/perf.go
Test data characteristics:
- 10% epics, 30% features, 60% tasks
- 4-level hierarchies (Epic → Feature → Task → Subtask)
- 20% cross-epic blocking dependencies
- Realistic status/priority/label distributions
Supports bd-l954 (Performance Testing Framework epic)
Closes bd-6ed8, bd-q59i
* perf: optimize GetReadyWork with compound index (20x speedup)
Add compound index on dependencies(depends_on_id, type, issue_id) to
eliminate performance bottleneck in GetReadyWork recursive CTE query.
Performance improvements (10K issue database):
- GetReadyWork: 752ms → 36.6ms (20.5x faster)
- Target: <50ms ✓ ACHIEVED
- 20K database: ~1500ms → 79.4ms (19x faster)
Benchmark infrastructure enhancements:
- Add dataset caching in /tmp/beads-bench-cache/ to avoid regenerating
10K-20K issues on every benchmark run (first run: ~2min, subsequent: <5s)
- Add progress logging during fixture generation (shows 10%, 20%... completion)
- Add database size logging (17.5 MB for 10K, 35.1 MB for 20K)
- Document rationale for only benchmarking large datasets (>10K issues)
- Add CPU/trace profiling with --profile flag for performance debugging
Schema changes:
- internal/storage/sqlite/schema.go: Add idx_dependencies_depends_on_type_issue
New files:
- internal/storage/sqlite/bench_helpers_test.go: Reusable benchmark setup with caching
- internal/storage/sqlite/sqlite_bench_test.go: Comprehensive benchmarks for critical operations
- Makefile: Convenient benchmark execution (make bench-quick, make bench)
Related:
- Resolves bd-5qim (optimize GetReadyWork performance)
- Builds on bd-6ed8 (fixture generator), bd-q59i (bd doctor --perf)
* perf: add WASM compilation cache to eliminate cold-start overhead
Configure wazero compilation cache for ncruces/go-sqlite3 to avoid
~220ms JIT compilation on every process start.
Cache configuration:
- Location: ~/.cache/beads/wasm/ (platform-specific via os.UserCacheDir)
- Automatic version management: wazero keys entries by its version
- Fallback: in-memory cache if directory creation fails
- No cleanup needed: old versions are harmless (~5-10MB each)
Performance impact:
- First run: ~220ms (populate cache)
- Subsequent runs: ~20ms (load from cache)
- Savings: ~200ms per cold start
Cache invalidation:
- Automatic when wazero version changes (upgrades use new cache dir)
- Manual cleanup: rm -rf ~/.cache/beads/wasm/ (safe to delete anytime)
This complements daemon mode:
- Daemon mode: eliminates startup cost by keeping process alive
- WASM cache: reduces startup cost for one-off commands or daemon restarts
Changes:
- internal/storage/sqlite/sqlite.go: Add init() with cache setup
* refactor: improve maintainability of performance testing code
Extract common patterns and eliminate duplication across benchmarks, fixture generation, and performance diagnostics. Replace magic numbers with explicit configuration to improve readability and make it easier to tune test parameters.
* docs: clarify profiling behavior and add missing documentation
Add explanatory comments for profiling setup to clarify why --profile
forces direct mode (captures actual database operations instead of RPC
overhead) and document the stopCPUProfile function's role in flushing
profile data to disk. Also fix gosec G104 linter warning by explicitly
ignoring Close() error during cleanup.
* fix: prevent bench-quick from running indefinitely
Added //go:build bench tags and skipped timeout-prone benchmarks to
prevent make bench-quick from running for hours.
Changes:
- Add //go:build bench tag to cycle_bench_test.go and compact_bench_test.go
- Skip Dense graph benchmarks (documented to timeout >120s)
- Fix compact benchmark prefix: bd- → bd (validation expects prefix without trailing dash)
Before: make bench-quick ran for 3.5+ hours (12,699s) before manual interrupt
After: make bench-quick completes in ~25 seconds
The Dense graph benchmarks are known to timeout and represent rare edge
cases that don't need optimization for typical workflows.
- Implement comprehensive schema probe in sqlite.New() that verifies all
expected tables and columns after migrations
- Add retry logic: if probe fails, retry migrations once
- Return clear fatal error with missing schema elements if probe still fails
- Enhance daemon version gating: refuse RPC if client has newer minor version
- Improve checkVersionMismatch messaging: verify schema before claiming upgrade
- Add schema compatibility check to bd doctor command
- Add comprehensive tests for schema probing
This prevents the silent migration failure bug where:
1. Migrations fail silently
2. Database queries fail with 'no such column' errors
3. Import logic misinterprets as 'not found' and tries INSERT
4. Results in cryptic UNIQUE constraint errors
Fixes#262
Amp-Thread-ID: https://ampcode.com/threads/T-0d7ae2c0-9f12-4f9b-85d1-1291488af150
Co-authored-by: Amp <amp@ampcode.com>
Without the 'file:' URI scheme prefix, SQLite treats query parameters
like '?mode=ro' as part of the filename instead of connection options.
This caused:
- Creation of bogus files named 'beads.db?mode=ro'
- Failure to read database version from metadata table
- bd doctor incorrectly reporting 'version pre-0.17.5 (very old)'
Fixed two sql.Open() calls in doctor.go:
- Line 317 (checkIssueIDs function)
- Line 403 (getDatabaseVersionFromPath function)
Now uses 'file:'+dbPath+'?mode=ro' pattern consistent with migrate.go.
Fixes#260
Amp-Thread-ID: https://ampcode.com/threads/T-ac9832d6-15e4-4e25-8027-5e8b640b190b
Co-authored-by: Amp <amp@ampcode.com>
* fix: Use correct SQLite driver name 'sqlite3' instead of 'sqlite'
The ncruces/go-sqlite3 driver registers as 'sqlite3', but doctor.go
and example code were using 'sqlite', causing 'unknown driver' errors.
This fix corrects all sql.Open() calls to use the proper driver name:
- cmd/bd/doctor.go: 6 instances fixed
- docs/EXTENDING.md: 2 documentation examples updated
- examples/bd-example-extension-go/: Fixed example code and README
Fixes#230
Amp-Thread-ID: https://ampcode.com/threads/T-1e8c5473-cb79-4457-be07-4517bfdb73f4
Co-authored-by: Amp <amp@ampcode.com>
* Revert CGO_ENABLED back to 0 for pure-Go SQLite driver
The ncruces/go-sqlite3 driver is pure-Go and doesn't require CGO.
The previous change to CGO_ENABLED=1 in commit f9771cd was an
attempted fix for #230, but the real issue was the driver name
mismatch ('sqlite' vs 'sqlite3'), which is now fixed.
Benefits of CGO_ENABLED=0:
- Simpler cross-compilation (no C toolchain required)
- Smaller binaries
- Faster builds
- Matches the intended design of the pure-Go driver
---------
Co-authored-by: Amp <amp@ampcode.com>
* feat: enhance bd doctor sync detection with count and prefix mismatch checks
Improves bd doctor to detect actual database-JSONL sync issues instead of relying only on file modification times:
Key improvements:
1. Count detection: Reports when database issue count differs from JSONL (e.g., "Count mismatch: database has 0 issues, JSONL has 61")
2. Prefix detection: Identifies prefix mismatches when majority of JSONL issues use different prefix than database config
3. Error handling: Returns errors from helper functions instead of silent failures, distinguishing "can't open DB" from "counts differ"
4. Query optimization: Single database connection for all checks (reduced from 3 opens to 1)
5. Better error reporting: Shows actual error details when database or JSONL can't be read
This addresses the core issue where bd doctor would incorrectly report "Database and JSONL are in sync" when the database was empty but JSONL contained issues (as happened in privacy2 project).
Tests:
- Added TestCountJSONLIssuesWithMalformedLines to verify malformed JSON handling
- Existing doctor tests still pass
- countJSONLIssues now returns error to indicate parsing issues
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: correct git hooks installation instructions in bd doctor
The original message referenced './examples/git-hooks/install.sh' which doesn't exist in user projects. This fix changes the message to point to the actual location in the beads GitHub repository:
Before: "Run './examples/git-hooks/install.sh' to install recommended git hooks"
After: "See https://github.com/steveyegge/beads/tree/main/examples/git-hooks for installation instructions"
This works for any project using bd, not just the beads repository itself.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add recovery suggestions when database fails but JSONL has issues
When bd doctor detects that the database cannot be opened/queried but the JSONL file contains issues, it now suggests the recovery command:
Fix: Run 'bd import -i issues.jsonl --rename-on-import' to recover issues from JSONL
This addresses the case where:
- Database is corrupted or inaccessible
- JSONL has all the issues backed up
- User needs a clear path to recover
The check now:
1. Reads JSONL first (doesn't depend on database)
2. If database fails but JSONL has issues, suggests recovery command
3. If database can be queried, continues with sync checks as before
Tested on privacy2 project which has 61 issues in JSONL but inaccessible database.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: support hash-based issue IDs in import rename
The import --rename-on-import flag was rejecting valid issue IDs with
hash-based suffixes (e.g., privacy-09ea) because the validation only
accepted numeric suffixes. Beads now generates and accepts base36-encoded
hash IDs, so update the validation to match.
Changes:
- Update isNumeric() to accept base36 characters (0-9, a-z)
- Update tests to reflect hash-based ID support
- Add gosec nolint comment for safe file path construction
Fixes the error: "cannot rename issue privacy-09ea: non-numeric suffix '09ea'"
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Switched from modernc.org/sqlite to ncruces/go-sqlite3 for WASM support
- Added WASM-specific stubs for daemon process management
- Created wasm/ directory with build.sh and Node.js runner
- WASM build succeeds (32MB bd.wasm)
- Node.js can load and execute the WASM module
- Next: Need to bridge Go file I/O to Node.js fs module
Related: bd-44d0, bd-8534, bd-c7eb
The --json flag was being ignored because doctor.go defined a local
flag that shadowed the persistent --json flag from main.go. The local
flag wasn't bound to the global jsonOutput variable, so it remained
false even when --json was passed.
Fixed by removing the duplicate local flag definition. The doctor
command now correctly uses the global persistent flag.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Adds checkGitHooks() function to verify recommended hooks are installed
- Checks for pre-commit, post-merge, and pre-push hooks
- Warns if hooks are missing with install instructions
- Shows up early in diagnostics (even if .beads/ missing)
- Includes comprehensive test coverage
- Filed bd-6049 for broken --json flag
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Renamed config.json to metadata.json to clarify purpose (database metadata)
- Fixed config.yaml/config.json conflict by making Viper explicitly load only config.yaml
- Added automatic migration from config.json to metadata.json on first read
- Fixed jsonOutput variable shadowing across 22 command files
- Updated bd init to create both metadata.json and config.yaml template
- Fixed 5 failing JSON output tests
- All tests passing
Resolves config file confusion and makes config.yaml work correctly.
Closes#178 (global flags), addresses config issues from #193
Amp-Thread-ID: https://ampcode.com/threads/T-e6ac8192-e18f-4ed7-83bc-4a5986718bb7
Co-authored-by: Amp <amp@ampcode.com>
- Added checkDependencyCycles() function using recursive CTE
- Integrated as Check #10 in runDiagnostics()
- Reports error status with count and fix suggestion if cycles found
- Updated doctor command documentation
- Fixes bd-3e3b
- Use path normalization (EvalSymlinks) to reliably match daemons across symlinks
- Check for stale sockets directly (catches cases where RPC failed)
- Detect multiple daemons for same workspace
- Enhanced error details for stale daemon detection
- Use global daemon registry instead of path-scoped discovery
Fixes#197: bd doctor was hardcoded to look for beads.db and didn't
check config.json for custom database names like beady.db.
Now both checkDatabaseVersion() and checkIDFormat() check config.json
first before falling back to the canonical database name.
* Add bd doctor command for installation health checks
Implements a comprehensive health check command similar to claude doctor
that validates beads installation and provides actionable recommendations.
Features:
- Installation check (.beads/ directory exists)
- Database version verification (compares with CLI version)
- ID format detection (hash-based vs sequential)
- CLI version check (fetches latest from GitHub)
- Storage type detection (SQLite vs JSONL-only mode)
- Tree-style output with color-coded warnings
- JSON output for scripting (--json flag)
- Actionable fix recommendations for each issue
Implementation improvements:
- Status constants instead of magic strings
- Semantic version comparison (fixes 0.10.0 vs 0.9.9 edge case)
- Documented defer pattern for intentional error ignore
- Comprehensive test coverage including version comparison edge cases
- Clean integration using slices.Contains for command list
Usage:
bd doctor # Check current directory
bd doctor /path/to/repo # Check specific repository
bd doctor --json # Machine-readable output
* Simplify bd doctor documentation in README
Reduce verbose health check section to 2 lines as requested.
* Fix bd doctor to handle JSONL-only mode for ID format check
When no SQLite database exists (JSONL-only mode), skip the ID format
check instead of showing an error. This prevents the confusing
'Unable to query issues' error when the installation is actually fine.
