The DB-JSONL Sync check was showing a false positive warning when wisps
(ephemeral issues) existed in the database. Wisps are intentionally
excluded from JSONL exports, so they shouldn't be counted when comparing
database and JSONL issue counts.
Changes:
- Updated CheckDatabaseJSONLSync to exclude ephemeral issues from count
using 'WHERE ephemeral = 0 OR ephemeral IS NULL'
- Added 'ephemeral' column to test database schema
- Added test case to verify ephemeral issues are excluded from count
- Updated TestCheckDatabaseJSONLSync_MoleculePrefix to include ephemeral column
This prevents false warnings like 'database has 92 issues, JSONL has 30'
when the difference is entirely due to wisps that should not be exported.
Co-authored-by: Roland Tritsch <roland@ailtir.com>
Problem:
- Diagnostic fix messages were restricted to single lines
- Inconsistent file path formatting in database sync issues
- Outdated cleanup command suggested in maintenance fixes
Solution:
- Implement multiline support for doctor fix messages
- Standardize issue file paths across database diagnostics
- Update maintenance fix instructions to use 'bd admin cleanup'
Impact:
- Clearer and more accurate recovery instructions for users
- Better readability of complex diagnostic output
Fixes: GH#1170, GH#1171, GH#1172
When bd sync fails mid-operation, the local JSONL can become stale while
the SQLite database has the correct state. Previously, bd doctor only
checked count and timestamp differences, missing cases where counts match
but issue statuses differ.
This adds content-level comparison to CheckDatabaseJSONLSync that:
- Compares issue statuses between DB and JSONL
- Samples up to 500 issues for performance on large databases
- Reports detailed mismatches (shows up to 3 examples)
- Suggests 'bd export' to fix the stale JSONL
Example detection:
Status mismatch: 1 issue(s) have different status in DB vs JSONL
Status mismatches detected:
test-1: DB=closed, JSONL=open
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The prefix mismatch check in bd doctor was reporting warnings when
issues were created via molecule workflows (bd mol pour), which
intentionally use a different prefix pattern (<base>-mol instead
of just <base>).
Added recognition of valid workflow prefix variants:
- <prefix>-mol (molecules from bd mol pour)
- <prefix>-wisp (ephemeral wisps)
- <prefix>-eph (ephemeral issues)
These are intentional prefix extensions for visual distinction, not
actual mismatches. The check now only warns for truly mismatched
prefixes (e.g., different project entirely).
Added comprehensive regression tests for all prefix variant cases.
Fixes#811
Generated with Claude Code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Post-merge cleanup of PR #805:
1. Extract duplicate error classification logic into classifyDatabaseError()
helper function (was duplicated in two places in database.go)
2. Fix semantic conflict between --force and --source=db flags:
- --force implies "database is broken, rebuild from JSONL"
- --source=db implies "use database as source of truth"
- These are contradictory; now errors with clear message
- --force with --source=auto or --source=jsonl works as expected
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Executed-By: beads/crew/dave
Rig: beads
Role: crew
Apply the same enhanced error classification to PRAGMA integrity_check
failures as we do for database open failures.
This ensures users see detailed, actionable recovery steps regardless of
which stage the corruption is detected (open vs integrity check).
Tested with real corruption scenarios - all error paths now provide
specific recovery guidance with exact commands.
Add comprehensive database corruption recovery capabilities to bd doctor.
## Changes
### New Command Flags
- --force: Force repair mode that bypasses database validation
- --source: Choose source of truth (auto/jsonl/db) for recovery
### Enhanced Error Classification
Improved CheckDatabaseIntegrity() to detect and classify:
- Database locked errors (suggests killing processes, removing locks)
- Invalid SQLite files (suggests JSONL recovery with exact commands)
- Migration/validation failures (orphaned dependencies, etc.)
- Generic database errors (context-aware suggestions)
Each error type provides:
- Specific diagnosis
- Step-by-step recovery instructions
- Appropriate command examples with new flags
### Force Recovery Implementation
New DatabaseCorruptionRecoveryWithOptions() function:
- Bypasses database validation when --force is used
- Supports explicit source of truth selection
- Auto-detects best recovery path when source=auto
- Comprehensive rollback on failure
- Uses --force --no-git-history in import during force mode
### Integration
Updated fix orchestration to pass force and source flags to recovery.
## Usage Examples
```bash
# Unopenable database with validation errors
bd doctor --fix --force --source=jsonl
# Choose specific source of truth
bd doctor --fix --source=jsonl # Trust JSONL
bd doctor --fix --source=db # Trust database
bd doctor --fix --source=auto # Auto-detect (default)
# Force recovery with auto-detection
bd doctor --fix --force
```
## Problem Solved
Before: When database had validation errors (orphaned dependencies,
foreign key violations), all bd commands failed in a catch-22 situation.
Could not open database to fix database. Users had to manually delete
database files and reinit.
After: bd doctor --fix --force detects unopenable databases, provides
clear recovery steps, and forces rebuild from JSONL even when database
validation fails.
## Backward Compatibility
- All new flags are optional with safe defaults
- --source defaults to 'auto' (existing behavior)
- --force is opt-in only
- Existing bd doctor behavior unchanged when flags not used
- DatabaseCorruptionRecovery() still exists for compatibility
Fixes: bd-pgza
Strip (bd-xxx), (gt-xxx) suffixes from code comments and changelog
entries. The descriptions remain meaningful without the ephemeral
issue IDs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The bug was that doctor's database checks used the local .beads/beads.db
file directly, while import followed the redirect to the actual database.
This caused confusing output showing 0 issues when the actual database
had hundreds. Fixed by using resolveBeadsDir() in all database checks.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Analysis found these commands are dead code:
- gt never calls `bd pin` - uses `bd update --status=pinned` instead
- Beads.Pin() wrapper exists but is never called
- bd hook functionality duplicated by gt mol status
- Code comment says "pinned field is cosmetic for bd hook visibility"
Removed:
- cmd/bd/pin.go
- cmd/bd/unpin.go
- cmd/bd/hook.go
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds automatic database recovery when bd doctor --fix detects corruption:
- Detects SQLite corruption (malformed database, SQLITE_CORRUPT errors)
- Backs up corrupted database before recovery attempt
- Rebuilds from JSONL if available (issues.jsonl, deletions.jsonl)
- Falls back to fresh database if JSONL unavailable
- Reports recovery results (issues imported, success/failure)
Recovery is triggered automatically by --fix when corruption is detected.
No manual intervention required.
feat(doctor): add count-based database size check
Adds CheckDatabaseSize (Check 29) that warns when closed issues exceed a configurable threshold (default: 5000). The check is informational only - no auto-fix since pruning is destructive.
Also improves fix guidance for sync-branch hook compatibility and legacy JSONL filename checks.
PR #724 by @rsnodgrass
- Modified fix.DBJSONLSync() to detect which direction to sync:
- If DB > JSONL: run 'bd export' to sync JSONL (DB has newer data)
- If JSONL > DB: run 'bd sync --import-only' to import (JSONL is source of truth)
- If equal but different timestamps: use file mtime to decide direction
- Updated CheckDatabaseJSONLSync() error messages to recommend correct fix direction:
- Shows different guidance based on whether DB or JSONL has more issues
- Added helper functions:
- countDatabaseIssues() to count issues in SQLite
- countJSONLIssues() to count issues in JSONL (local, avoids circular import)
- Added tests for countJSONLIssues() with edge cases
Fixes issue where 'bd doctor --fix' would recommend 'bd sync --import-only'
when DB > JSONL, which would be a no-op since JSONL hasn't changed.
* test(doctor): add comprehensive tests for fix and check functions
Add edge case tests, e2e tests, and improve test coverage for:
- database_test.go: database integrity and sync checks
- git_test.go: git hooks, merge driver, sync branch tests
- gitignore_test.go: gitignore validation
- prefix_test.go: ID prefix handling
- fix/fix_test.go: fix operations
- fix/e2e_test.go: end-to-end fix scenarios
- fix/fix_edge_cases_test.go: edge case handling
* docs: add testing philosophy and anti-patterns guide
- Create TESTING_PHILOSOPHY.md covering test pyramid, priority matrix,
what NOT to test, and 5 anti-patterns with code examples
- Add cross-reference from README_TESTING.md
- Document beads-specific guidance (well-covered areas vs gaps)
- Include target metrics (test-to-code ratio, execution time targets)
* chore: revert .beads/ to upstream/main state
* refactor(doctor): add category grouping and Ayu theme colors
- Add Category field to DoctorCheck for organizing checks by type
- Define category constants: Core, Git, Runtime, Data, Integration, Metadata
- Update thanks command to use shared Ayu color palette from internal/ui
- Simplify test fixtures by removing redundant test cases
* fix(doctor): prevent test fork bomb and fix test failures
- Add ErrTestBinary guard in getBdBinary() to prevent tests from
recursively executing the test binary when calling bd subcommands
- Update claude_test.go to use new check names (CLI Availability,
Prime Documentation)
- Fix syncbranch test path comparison by resolving symlinks
(/var vs /private/var on macOS)
- Fix permissions check to use exact comparison instead of bitmask
- Fix UntrackedJSONL to use git commit --only to preserve staged changes
- Fix MergeDriver edge case test by making both .git dir and config
read-only
- Add skipIfTestBinary helper for E2E tests that need real bd binary
* test(doctor): skip read-only config test in CI environments
GitHub Actions containers may have CAP_DAC_OVERRIDE or similar
capabilities that allow writing to read-only files, causing
the test to fail. Skip the test when CI=true or GITHUB_ACTIONS=true.
The schema_probe.go and doctor/database.go had different column lists
for validating database schemas. Neither included the pinned column
added in migration 023, causing:
- Doctor to report "All required tables and columns present" on
databases missing the pinned column
- Potential failures when using `gt mail send` on databases without
the pinned column
Changes:
- Add pinned, sender, ephemeral to doctor's criticalChecks
- Add metadata, thread_id to dependencies check
- Add all missing columns to schema_probe.go's expectedSchema:
source_repo, close_reason, deleted_at, deleted_by, delete_reason,
original_type, sender, ephemeral, pinned
Fixes: beads-9yc
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(doctor): split doctor.go into modular package files
Split the 3,171-line doctor.go into logical sub-files within the
cmd/bd/doctor/ package, reducing the main file to 834 lines (74% reduction).
New package structure:
- types.go: DoctorCheck struct and status constants
- installation.go: CheckInstallation, CheckMultipleDatabases, CheckPermissions
- git.go: CheckGitHooks, CheckMergeDriver, CheckSyncBranch* checks
- database.go: CheckDatabaseVersion, CheckSchemaCompatibility, CheckDatabaseJSONLSync
- version.go: CheckCLIVersion, CheckMetadataVersionTracking, CompareVersions
- integrity.go: CheckIDFormat, CheckDependencyCycles, CheckTombstones
- daemon.go: CheckDaemonStatus, CheckVersionMismatch
- quick.go: Quick checks for sync-branch and hooks
Updated tests to use exported doctor.CheckXxx() functions and
doctor.StatusXxx constants.
* fix(doctor): suppress gosec G204 false positives
Add #nosec G204 comments to exec.Command calls in CheckSyncBranchHealth
where variables come from trusted sources (config files or hardcoded
values like "main"/"master"/"origin"), not untrusted user input.
