From 9db8efaa5c2ba13599faf8769b6341293128d482 Mon Sep 17 00:00:00 2001 From: matt wilkie Date: Sun, 21 Dec 2025 11:25:57 -0700 Subject: [PATCH] docs: update docs.md for bidirectional DBJSONLSync fix (bd-68e4) --- cmd/bd/doctor/fix/docs.md | 59 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/cmd/bd/doctor/fix/docs.md b/cmd/bd/doctor/fix/docs.md index 0ed36bdc..cedb8531 100644 --- a/cmd/bd/doctor/fix/docs.md +++ b/cmd/bd/doctor/fix/docs.md @@ -8,16 +8,53 @@ The `cmd/bd/doctor/fix` directory contains automated remediation functions for i ### How it fits into the larger codebase -- **Integration with Doctor Detection**: The `@/cmd/bd/doctor.go` command runs checks to identify workspace problems, then calls functions from this package when `--fix` flag is used. The doctor command is the orchestrator that determines which issues exist and which fixes to apply. +- **Integration with Doctor Detection**: The `@/cmd/bd/doctor.go` command runs checks to identify workspace problems, then calls functions from this package when `--fix` flag is used. The `CheckDatabaseJSONLSync()` function in `@/cmd/bd/doctor/database.go` (lines 299-486) detects sync issues and provides direction-specific guidance about which fix to run. When DB count differs from JSONL count, it now recommends `bd doctor --fix` to run `DBJSONLSync()` with the appropriate direction. -- **Dependency on Core Libraries**: The fix functions use core libraries like `@/internal/deletions` (for reading/writing deletion manifests), `@/internal/types` (for issue data structures), and git operations via `exec.Command`. +- **Dependency on Core Libraries**: The fix functions use core libraries like `@/internal/deletions` (for reading/writing deletion manifests), `@/internal/types` (for issue data structures), `@/internal/configfile` (for database path resolution), and git operations via `exec.Command`. -- **Data Persistence Points**: Each fix module directly modifies persistent workspace state: deletions manifest, database files, JSONL files, and git branch configuration. Changes are written to disk and persisted in the git repository. +- **Data Persistence Points**: Each fix module directly modifies persistent workspace state: deletions manifest, database files, JSONL files, and git branch configuration. Changes are written to disk and persisted in the git repository. The sync fix is unique in that it delegates persistence to `bd export` or `bd sync --import-only` commands. - **Deletion Tracking Architecture**: The deletions manifest (`@/internal/deletions/deletions.go`) is an append-only log tracking issue deletions. The fix in `deletions.go` is critical to maintaining the integrity of this log by preventing tombstones from being incorrectly re-added to it after `bd migrate-tombstones` runs. - **Tombstone System**: The fix works in concert with the tombstone system (`@/internal/types/types.go` - `Status == StatusTombstone`). Tombstones represent soft-deleted issues that contain deletion metadata. The fix prevents tombstones from being confused with actively deleted issues during deletion hydration. +- **Database Configuration Management**: The sync fix uses `@/internal/configfile.Load()` to support both canonical and custom database paths, enabling workspaces with non-standard database locations (via `metadata.json`) to be synced correctly. + +**Database-JSONL Sync** (`sync.go`): + +The `DBJSONLSync()` function fixes synchronization issues between the SQLite database and JSONL export files by detecting data direction and running the appropriate sync command: + +1. **Bidirectional Detection** (lines 23-97): + - Counts issues in both database (via SQL query) and JSONL file (via line-by-line JSON parsing) + - Determines sync direction based on issue counts: + - If `dbCount > jsonlCount`: DB has newer data → runs `bd export` to sync JSONL + - If `jsonlCount > dbCount`: JSONL has newer data → runs `bd sync --import-only` to import + - If counts equal but timestamps differ: Uses file modification times to decide direction + - This replaces the previous unidirectional approach that could leave users stuck when DB was the source of truth + +2. **Database Path Resolution** (lines 32-37): + - Uses `configfile.Load()` to check for custom database paths in `metadata.json` + - Falls back to canonical database name (`beads.CanonicalDatabaseName`) + - Supports both current and legacy database configurations + +3. **JSONL File Discovery** (lines 39-48): + - Checks for both canonical (`issues.jsonl`) and legacy (`beads.jsonl`) JSONL file names + - Supports workspaces that migrated from one naming convention to another + - Returns early if either database or JSONL is missing (nothing to sync) + +4. **Helper Functions**: + - `countDatabaseIssues()` (lines 124-139): Opens SQLite database and queries `COUNT(*) FROM issues` + - `countJSONLIssues()` (lines 141-174): Iterates through JSONL file line-by-line, parsing JSON and counting valid issues with IDs. Skips malformed JSON lines gracefully. + +5. **Command Execution** (lines 106-120): + - Gets bd binary path safely via `getBdBinary()` to prevent fork bombs in tests + - Executes `bd export` or `bd sync --import-only` with workspace directory as working directory + - Streams stdout/stderr to user for visibility + +**Problem Solved (bd-68e4)**: + +Previously, when the database contained more issues than the JSONL export, the doctor would recommend `bd sync --import-only`, which imports JSONL into DB. Since JSONL hadn't changed and the database had newer data, this command was a no-op (0 created, 0 updated), leaving users unable to sync their JSONL file with the database. The bidirectional detection now recognizes this case and runs `bd export` instead. + ### Core Implementation **Deletions Manifest Hydration** (`deletions.go`): @@ -52,6 +89,22 @@ The `cmd/bd/doctor/fix` directory contains automated remediation functions for i - Prefix must be alphanumeric with underscores, suffix must be base36 hash or number with optional dots for child issues - Used to filter out false positives when parsing JSON +**Test Coverage** (`fix_test.go`): + +The test file includes comprehensive coverage for the sync functionality: + +- **TestCountJSONLIssues**: Tests the `countJSONLIssues()` helper with: + - Empty JSONL files (returns 0) + - Valid issues in JSONL (correct count) + - Mixed valid and invalid JSON lines (counts only valid issues) + - Nonexistent files (returns error) + +- **TestDBJSONLSync_Validation**: Tests validation logic: + - Returns without error when no database exists (nothing to sync) + - Returns without error when no JSONL exists (nothing to sync) + +- **TestDBJSONLSync_MissingDatabase**: Validates graceful handling when only JSONL exists + **Test Coverage** (`deletions_test.go`): The test file covers edge cases and validates the bd-in7q fix: