diff --git a/.beads/beads.jsonl b/.beads/beads.jsonl index d35831fd..af42802e 100644 --- a/.beads/beads.jsonl +++ b/.beads/beads.jsonl @@ -53,7 +53,14 @@ {"id":"bd-146","title":"Add configurable SortPolicy to GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.","design":"See SORT_POLICY_DESIGN.md for complete design.\n\nImplementation summary:\n1. Add SortPolicy type and constants (hybrid, priority, oldest)\n2. Add SortPolicy field to WorkFilter \n3. Implement buildOrderByClause() to generate SQL based on policy\n4. Default to hybrid for backwards compatibility\n5. Add --sort flag to bd ready command\n\nThis enables autonomous execution systems (like VC) to use strict priority ordering while preserving the current hybrid behavior for interactive use.","acceptance_criteria":"Unit tests verify each policy generates correct ORDER BY. Integration tests verify priority, hybrid, and oldest policies select issues in expected order. CLI bd ready --sort priority works. Empty SortPolicy defaults to hybrid (backwards compatible).","status":"closed","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:44:27.907777-07:00","updated_at":"2025-10-25T18:50:52.816914-07:00","closed_at":"2025-10-25T18:50:52.816914-07:00"} {"id":"bd-147","title":"Implement configurable sort policy for GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different sorting strategies:\n- hybrid (default): Recent issues by priority, old by age\n- priority: Strict priority ordering for autonomous execution\n- oldest: Backlog clearing mode\n\nSolves issue where VC executor selects low-priority work instead of critical P0 work.","design":"See SORT_POLICY_DESIGN.md for complete design spec including:\n- Type definitions and constants\n- SQL ORDER BY clause generation\n- Testing strategy with test cases\n- CLI integration with --sort flag\n- Migration plan and backwards compatibility","acceptance_criteria":"- SortPolicy type added to types.go\n- buildOrderByClause() implemented in ready.go\n- Unit tests for all 3 policies pass\n- Integration tests verify priority selection order\n- bd ready --sort priority|oldest|hybrid works\n- Empty SortPolicy defaults to hybrid (backwards compatible)\n- Documentation updated in README.md and WORKFLOW.md","status":"closed","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:46:44.35198-07:00","updated_at":"2025-10-25T18:53:02.258745-07:00","closed_at":"2025-10-25T18:53:02.258745-07:00"} {"id":"bd-148","title":"Add configurable SortPolicy to GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.","design":"See SORT_POLICY_DESIGN.md for complete design.\n\nImplementation summary:\n1. Add SortPolicy type and constants (hybrid, priority, oldest)\n2. Add SortPolicy field to WorkFilter \n3. Implement buildOrderByClause() to generate SQL based on policy\n4. Default to hybrid for backwards compatibility\n5. Add --sort flag to bd ready command\n\nThis enables autonomous execution systems (like VC) to use strict priority ordering while preserving the current hybrid behavior for interactive use.","acceptance_criteria":"Unit tests verify each policy generates correct ORDER BY. Integration tests verify priority, hybrid, and oldest policies select issues in expected order. CLI bd ready --sort priority works. Empty SortPolicy defaults to hybrid (backwards compatible).","status":"open","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:51:11.560979-07:00","updated_at":"2025-10-25T18:51:11.560979-07:00"} +{"id":"bd-149","title":"Enforce daemon singleton per workspace with file locking","description":"Agent in ~/src/wyvern discovered 4 simultaneous daemon processes running, causing infinite directory recursion (.beads/.beads/.beads/...). Each daemon used relative paths and created nested .beads/ directories.\n\nRoot cause: No singleton enforcement. Multiple `bd daemon` processes can start in same workspace.\n\nExpected: One daemon per workspace (each workspace = separate .beads/ dir with bd.sock)\nActual: Multiple daemons can run simultaneously in same workspace\n\nNote: Separate git clones = separate workspaces = separate daemons (correct). Git worktrees share .beads/ and have known limitations (documented, use --no-daemon).","design":"Use flock (file locking) on daemon socket or database file to enforce singleton:\n\n1. On daemon start, attempt exclusive lock on .beads/bd.sock or .beads/daemon.lock\n2. If lock held by another process, refuse to start (exit with clear error)\n3. Hold lock for lifetime of daemon process\n4. Release lock on daemon shutdown\n\nAlternative: Use PID file with stale detection (check if PID is still running)\n\nImplementation location: Daemon startup code in cmd/bd/ or internal/daemon/","acceptance_criteria":"1. Starting second daemon process in same workspace fails with clear error\n2. Test: Start daemon, attempt second start, verify failure\n3. Killing daemon releases lock, allowing new daemon to start\n4. No infinite .beads/ directory recursion possible\n5. Works correctly with auto-start mechanism","status":"open","priority":0,"issue_type":"bug","created_at":"2025-10-25T22:23:59.433044-07:00","updated_at":"2025-10-25T22:23:59.433044-07:00"} {"id":"bd-15","title":"Make merge command idempotent for safe retry after partial failures","description":"The merge command currently performs 3 operations without an outer transaction:\n1. Migrate dependencies from source → target\n2. Update text references across all issues\n3. Close source issues\n\nIf merge fails mid-operation (network issue, daemon crash, etc.), a retry will fail or produce incorrect results because some operations already succeeded.\n\n**Goal:** Make merge idempotent so retrying after partial failure is safe and completes the remaining work.\n\n**Idempotency checks needed:**\n- Skip dependency migration if target already has the dependency\n- Skip text reference updates if already updated\n- Skip closing source issues if already closed\n- Report which operations were skipped vs performed\n\n**Example output:**\n```\n✓ Merged 2 issue(s) into bd-63\n - Dependencies: 3 migrated, 2 already existed\n - Text references: 5 updated, 0 already correct\n - Source issues: 1 closed, 1 already closed\n```\n\n**Related:** bd-115 originally requested transaction support, but idempotency is a better solution for this use case since individual operations are already atomic.","design":"Current merge code already has some idempotency:\n- Dependency migration checks `alreadyExists` before adding (line ~145-151 in merge.go)\n- Text reference updates are naturally idempotent (replacing bd-X with bd-Y twice has same result)\n\nMissing idempotency:\n- CloseIssue fails if source already closed\n- Error messages don't distinguish \"already done\" from \"real failure\"\n\nImplementation:\n1. Check source issue status before closing - skip if already closed\n2. Track which operations succeeded/skipped\n3. Return detailed results for user visibility\n4. Consider adding --dry-run output showing what would be done vs skipped","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165434-07:00","updated_at":"2025-10-24T13:51:54.437619-07:00","closed_at":"2025-10-22T11:56:36.526276-07:00"} +{"id":"bd-150","title":"Implement exclusive lock protocol for daemon/external tool coexistence","description":"VC (VibeCoder) cannot coexist with bd daemon managing the same database. VC needs deterministic execution and directly manages JSONL sync, but daemon's auto-flush creates timing conflicts and nondeterministic behavior.\n\nCurrent workaround (vc-195): VC kills ALL bd daemon processes with pkill, which is too aggressive and prevents developers from using daemon for their own workflow.\n\nSolution: Implement lock file protocol (.beads/.exclusive-lock) that allows applications to claim exclusive management of a database. bd daemon respects these locks and skips locked databases.\n\nSee VC_DAEMON_EXCLUSION_PROTOCOL.md for full specification.","design":"Lock File Protocol:\n- File: .beads/.exclusive-lock (JSON format)\n- Contains: holder name, PID, hostname, timestamp, version\n- Created when external tool (VC) starts\n- Removed on clean shutdown\n- Stale lock detection: if PID doesn't exist, lock is removed\n\nbd daemon behavior:\n1. Before touching any database, check for .exclusive-lock\n2. If lock exists and valid (PID alive): skip database in sync cycle\n3. If lock is stale (PID dead): remove lock and proceed\n4. If lock is malformed: log warning and skip (fail-safe)\n\nExternal tool behavior:\n1. On startup: create .exclusive-lock with process info\n2. If lock exists, check if stale; fail if another process is running\n3. On shutdown: remove .exclusive-lock\n\nBenefits:\n- Clean separation, no process killing\n- Developer-friendly (can run daemon for other projects)\n- Fail-safe with stale lock detection\n- Extensible for other tools beyond VC","acceptance_criteria":"1. bd daemon skips databases with valid exclusive locks\n2. bd daemon cleans up stale locks automatically\n3. Lock file format documented and implemented\n4. Integration tests show daemon + external tool coexistence\n5. Documentation updated with protocol specification","status":"open","priority":1,"issue_type":"epic","created_at":"2025-10-25T22:43:08.994891-07:00","updated_at":"2025-10-25T22:43:08.994891-07:00"} +{"id":"bd-151","title":"Add ExclusiveLock struct and JSON marshaling","description":"Create Go struct to represent exclusive lock file format with JSON marshaling support.\n\nFields needed:\n- Holder (string): name of lock holder (e.g., \"vc-executor\")\n- PID (int): process ID\n- Hostname (string): hostname where process is running\n- StartedAt (time.Time): when lock was acquired\n- Version (string): version of lock holder\n\nShould support both Marshal and Unmarshal for reading/writing lock files.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:18.536901-07:00","updated_at":"2025-10-25T22:43:18.536901-07:00","dependencies":[{"issue_id":"bd-151","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:18.538184-07:00","created_by":"daemon"}]} +{"id":"bd-152","title":"Implement isProcessAlive() helper for PID validation","description":"Implement helper function to check if a process is alive given PID and hostname.\n\nLogic:\n- If hostname != current host: return true (can't verify remote, assume alive)\n- If hostname == current host: check if PID exists using os.FindProcess + Signal(0)\n- Return true if process exists, false otherwise\n\nNeeded for stale lock detection.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:26.074997-07:00","updated_at":"2025-10-25T22:43:26.074997-07:00","dependencies":[{"issue_id":"bd-152","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:28.313879-07:00","created_by":"daemon"}]} +{"id":"bd-153","title":"Implement shouldSkipDatabase() to check for exclusive locks","description":"Add shouldSkipDatabase() function to daemon that checks for .beads/.exclusive-lock before processing a database.\n\nLogic:\n1. Check if .beads/.exclusive-lock exists\n2. If not: return false (proceed with database)\n3. If yes: read and parse JSON\n4. If malformed: log warning and skip (fail-safe)\n5. If valid: check if PID is alive\n6. If stale (PID dead): remove lock file and proceed\n7. If active (PID alive): log skip message and return true\n\nDepends on ExclusiveLock struct and isProcessAlive() helper.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:34.561247-07:00","updated_at":"2025-10-25T22:43:34.561247-07:00","dependencies":[{"issue_id":"bd-153","depends_on_id":"bd-151","type":"blocks","created_at":"2025-10-25T22:43:36.6504-07:00","created_by":"daemon"},{"issue_id":"bd-153","depends_on_id":"bd-152","type":"blocks","created_at":"2025-10-25T22:43:38.730862-07:00","created_by":"daemon"},{"issue_id":"bd-153","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:40.602161-07:00","created_by":"daemon"}]} +{"id":"bd-154","title":"Integrate exclusive lock check into daemon sync cycle","description":"Integrate shouldSkipDatabase() into daemon's sync cycle to respect exclusive locks.\n\nBefore processing each database:\n1. Call shouldSkipDatabase(beadsDir)\n2. If true: skip all operations (export, import, commit, push)\n3. If false: proceed with normal sync\n\nAdd appropriate logging:\n- \"Skipping .beads/foo.db (locked by vc-executor, PID 12345)\" when skipping\n- \"Removed stale lock at .beads/foo.db (PID 12345 not found)\" when cleaning up\n\nDepends on shouldSkipDatabase() implementation.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:46.221736-07:00","updated_at":"2025-10-25T22:43:46.221736-07:00","dependencies":[{"issue_id":"bd-154","depends_on_id":"bd-153","type":"blocks","created_at":"2025-10-25T22:43:48.204682-07:00","created_by":"daemon"},{"issue_id":"bd-154","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:50.283951-07:00","created_by":"daemon"}]} +{"id":"bd-155","title":"Add unit tests for exclusive lock detection","description":"Add unit tests for exclusive lock functionality:\n\nTest cases:\n1. No lock file exists -\u003e shouldSkipDatabase returns false\n2. Valid lock with alive PID -\u003e shouldSkipDatabase returns true\n3. Stale lock with dead PID -\u003e removes lock, returns false\n4. Malformed lock JSON -\u003e logs warning, returns true (fail-safe)\n5. Lock with different hostname -\u003e assumes alive, returns true\n6. isProcessAlive() correctly detects running/dead processes\n\nUse test helpers to create lock files with specific PIDs.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:57.616-07:00","updated_at":"2025-10-25T22:43:57.616-07:00"} {"id":"bd-16","title":"Global daemon should warn/reject --auto-commit and --auto-push","description":"When user runs 'bd daemon --global --auto-commit', it's unclear which repo the daemon will commit to (especially after fixing bd-62 where global daemon won't open a DB).\n\nOptions:\n1. Warn and ignore the flags in global mode\n2. Error out with clear message\n\nLine 87-91 already checks autoPush, but should skip check entirely for global mode. Add user-friendly messaging about flag incompatibility.","status":"closed","priority":3,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165645-07:00","updated_at":"2025-10-24T13:51:54.437812-07:00","closed_at":"2025-10-17T23:04:30.223432-07:00"} {"id":"bd-17","title":"Add cross-repo issue references (future enhancement)","description":"Support referencing issues across different beads repositories. Useful for tracking dependencies between separate projects.\n\nProposed syntax:\n- Local reference: bd-63 (current behavior)\n- Cross-repo by path: ~/src/other-project#bd-456\n- Cross-repo by workspace name: @project2:bd-789\n\nUse cases:\n1. Frontend project depends on backend API issue\n2. Shared library changes blocking multiple projects\n3. System administrator tracking work across machines\n4. Monorepo with separate beads databases per component\n\nImplementation challenges:\n- Storage layer needs to query external databases\n- Dependency resolution across repos\n- What if external repo not available?\n- How to handle in JSONL export/import?\n- Security: should repos be able to read others?\n\nDesign questions to resolve first:\n1. Read-only references vs full cross-repo dependencies?\n2. How to handle repo renames/moves?\n3. Absolute paths vs workspace names vs git remotes?\n4. Should bd-38 auto-discover related repos?\n\nRecommendation: \n- Gather user feedback first\n- Start with read-only references\n- Implement as plugin/extension?\n\nContext: This is mentioned in bd-38 as approach #2. Much more complex than daemon multi-repo approach. Only implement if there's strong user demand.\n\nPriority: Backlog (4) - wait for user feedback before designing","status":"closed","priority":4,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165857-07:00","updated_at":"2025-10-24T13:51:54.438011-07:00","closed_at":"2025-10-20T22:00:31.966891-07:00"} {"id":"bd-18","title":"Make beads reusable as a Go library for external projects like vc","description":"Currently beads is only usable as a CLI tool. We want to use beads as a library in other Go projects like ~/src/vc so they can programmatically manage issues without shelling out to the bd CLI.\n\nGoals:\n- Export public API from internal packages\n- Document Go package usage\n- Provide examples of programmatic usage\n- Ensure vc can import and use beads storage layer directly\n\nUse case: The vc project needs issue tracking and wants to use beads as an embedded library rather than as a separate CLI tool.","notes":"UnderlyingDB() method implemented and tested. Core functionality complete. Still needs documentation updates (bd-26) and lifecycle safety enhancements (bd-25).","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-22T12:27:30.35968-07:00","updated_at":"2025-10-24T13:51:54.438187-07:00","closed_at":"2025-10-22T19:46:09.362533-07:00"} diff --git a/DATABASE_REINIT_BUG.md b/DATABASE_REINIT_BUG.md deleted file mode 100644 index 7169ad69..00000000 --- a/DATABASE_REINIT_BUG.md +++ /dev/null @@ -1,484 +0,0 @@ -# Database Re-initialization Bug Investigation - -**Date**: 2024-10-24 -**Severity**: P0 Critical -**Status**: Under Investigation - -## Problem Statement - -When `.beads/` directory is removed and daemon auto-starts, it creates an **empty database** instead of importing from git-tracked JSONL file. This causes silent data loss. - -## What Happened - -1. **Initial State**: ~/src/fred/beads had polluted database with 202 issues -2. **Action Taken**: Removed `.beads/` directory to clean pollution: `rm -rf .beads/` -3. **Session Restart**: Amp session restarted, working directory: `/Users/stevey/src/fred/beads` -4. **Auto-Init Triggered**: Daemon auto-started and created fresh database -5. **Result**: Empty database (0 issues) despite `.beads/beads.jsonl` in git with 111 issues - -## Root Cause Analysis - -### Key Observations - -1. **File Naming Confusion** - - Git history shows rename: `issues.jsonl → beads.jsonl` (commit d1d3fcd) - - Daemon created new `issues.jsonl` (empty) - - Auto-import may be looking for wrong filename - -2. **Auto-Import Failed** - - `bd init` ran successfully - - Auto-import from git did NOT trigger - - Expected behavior: should import from `.beads/beads.jsonl` in git - -3. **Daemon Startup Sequence** - ``` - [2025-10-24 13:19:42] Daemon started - [2025-10-24 13:19:42] Using database: /Users/stevey/src/fred/beads/.beads/bd.db - [2025-10-24 13:19:42] Database opened - [2025-10-24 13:19:42] Exported to JSONL (exported 0 issues to empty file) - ``` - -4. **Multiple Database Problem** - - Three separate beads databases detected: - - `~/src/beads/.beads/bd.db` (4.2MB, 112 issues) ✅ CORRECT - - `~/src/fred/beads/.beads/bd.db` (155KB, 0 issues) ❌ EMPTY - - `~/src/original/beads/.beads/bd.db` ❌ UNKNOWN STATE - -## Expected Behavior - -When `.beads/` directory is missing but git has tracked JSONL: - -1. `bd init` should detect git-tracked JSONL file -2. Auto-import should trigger immediately -3. Database should be populated from git history -4. User should see: "Imported N issues from git" - -## Actual Behavior - -1. `bd init` creates empty database -2. Auto-import does NOT trigger -3. Database remains empty (0 issues) -4. Silent data loss - user unaware issues are missing - -## Impact - -- **Silent Data Loss**: Users lose entire issue database without warning -- **Multi-Workspace Confusion**: Per-project daemons don't handle missing DB correctly -- **Git Sync Broken**: Auto-import from git not working as expected -- **User Trust**: Critical failure mode that breaks core workflow - -## Recovery Steps Taken - -1. Restored from git: `git restore .beads/beads.jsonl` ❌ File already in git, not in working tree -2. Extracted from git history: `git show HEAD:.beads/beads.jsonl > /tmp/backup.jsonl` -3. Manual import with collision resolution: `bd import -i /tmp/backup.jsonl --resolve-collisions` -4. Final state: 194 issues recovered (had stale backup) - -## Correct Recovery (Final) - -1. Removed bad database: `rm -f .beads/beads.db` -2. Git pull to get latest: `git pull origin main` (got 111 issues from ~/src/beads) -3. Re-init with correct prefix: `bd init --prefix bd` -4. Import from git-tracked JSONL: `bd import -i .beads/beads.jsonl` -5. ✅ Result: 112 issues (111 + external_ref epic from main database) - -## Technical Investigation Needed - -### 1. Auto-Import Logic -- Where is auto-import triggered? (`bd init` command? daemon startup?) -- What file does it look for? (`issues.jsonl` vs `beads.jsonl`) -- Why didn't it run when `.beads/` was missing? - -### 2. Daemon Initialization -- Should daemon auto-import on first startup? -- Should daemon detect missing database and import from git? -- Per-project daemon handling when DB missing - -### 3. File Naming -- When did `issues.jsonl → beads.jsonl` rename happen? -- Are all code paths updated to use correct filename? -- Is auto-import looking for old filename? - -### 4. Git Integration -- Should `bd init` check for tracked JSONL in git? -- Should init fail if git has JSONL but DB is empty after init? -- Add warning: "JSONL found in git but not imported"? - -## Proposed Fixes (Oracle-Reviewed) - -### Fix A: checkGitForIssues() Filename Detection (P0, Simple, <1h) - -**Current Code** (autoimport.go:70-76): -```go -relPath, err := filepath.Rel(gitRoot, filepath.Join(beadsDir, "issues.jsonl")) -``` - -**Fixed Code**: -```go -// Try canonical JSONL filenames in precedence order -relBeads, err := filepath.Rel(gitRoot, beadsDir) -if err != nil { - return 0, "" -} - -candidates := []string{ - filepath.Join(relBeads, "beads.jsonl"), - filepath.Join(relBeads, "issues.jsonl"), -} - -for _, relPath := range candidates { - cmd := exec.Command("git", "show", fmt.Sprintf("HEAD:%s", relPath)) - output, err := cmd.Output() - if err == nil && len(output) > 0 { - lines := bytes.Count(output, []byte("\n")) - return lines, relPath - } -} - -return 0, "" -``` - -**Impact**: Auto-import will now detect beads.jsonl in git - ---- - -### Fix B: findJSONLPath() Consults Git HEAD (P0, Simple-Medium, 1-2h) - -**Current Code** (main.go:898-912): -```go -func findJSONLPath() string { - jsonlPath := beads.FindJSONLPath(dbPath) - // Creates directory but doesn't check git - return jsonlPath -} -``` - -**Fixed Code**: -```go -func findJSONLPath() string { - // First check for existing local JSONL files - jsonlPath := beads.FindJSONLPath(dbPath) - - dbDir := filepath.Dir(dbPath) - - // If local file exists, use it - if _, err := os.Stat(jsonlPath); err == nil { - return jsonlPath - } - - // No local JSONL - check git HEAD for tracked filename - if gitJSONL := checkGitForJSONLFilename(); gitJSONL != "" { - jsonlPath = filepath.Join(dbDir, filepath.Base(gitJSONL)) - } - - // Ensure directory exists - if err := os.MkdirAll(dbDir, 0755); err == nil { - // Verify we didn't pick the wrong file - // ...error checking... - } - - return jsonlPath -} -``` - -**Impact**: Daemon/CLI will export to beads.jsonl (not issues.jsonl) when git tracks beads.jsonl - ---- - -### Fix C: Init Safety Check (P0, Simple, <1h) - -**Location**: cmd/bd/init.go after line 150 - -**Add After Import Attempt**: -```go -// Safety check: verify import succeeded -stats, err := store.GetStatistics(ctx) -if err == nil && stats.TotalIssues == 0 { - // DB empty after init - check if git has issues we failed to import - recheck, _ := checkGitForIssues() - if recheck > 0 { - fmt.Fprintf(os.Stderr, "\n❌ ERROR: Database empty but git has %d issues!\n", recheck) - fmt.Fprintf(os.Stderr, "Auto-import failed. Manual recovery:\n") - fmt.Fprintf(os.Stderr, " git show HEAD:%s | bd import -i /dev/stdin\n", jsonlPath) - fmt.Fprintf(os.Stderr, "Or:\n") - fmt.Fprintf(os.Stderr, " bd import -i %s\n", jsonlPath) - os.Exit(1) - } -} -``` - -**Impact**: Prevents silent data loss by failing loudly with recovery instructions - ---- - -### Fix D: Daemon Startup Import (P1, Simple, <1h) - -**Location**: cmd/bd/daemon.go after DB open (around line 914) - -**Add After Database Open**: -```go -// Check for empty DB with issues in git -ctx := context.Background() -stats, err := store.GetStatistics(ctx) -if err == nil && stats.TotalIssues == 0 { - issueCount, jsonlPath := checkGitForIssues() - if issueCount > 0 { - log(fmt.Sprintf("Empty database but git has %d issues, importing...", issueCount)) - if err := importFromGit(ctx, dbPath, store, jsonlPath); err != nil { - log(fmt.Sprintf("Warning: startup import failed: %v", err)) - } else { - log(fmt.Sprintf("Successfully imported %d issues from git", issueCount)) - } - } -} -``` - -**Impact**: Daemon auto-recovers from empty DB on startup - -### Medium Term (P1) -1. **Multiple database warning** (bd-112) - - Detect multiple `.beads/` in workspace hierarchy - - Warn user on startup - - Prevent accidental database pollution - -2. **Better error messages** - - `bd init`: "Warning: found beads.jsonl in git with N issues" - - `bd stats`: "Warning: database empty but git has tracked JSONL" - - Guide user to recovery path - -### Implementation Refinements (Critical) - -**Fix B Missing Helper Function**: -The oracle's Fix B pseudocode calls `checkGitForJSONLFilename()` which doesn't exist. Need to add: -```go -// checkGitForJSONLFilename returns just the filename from git HEAD check -func checkGitForJSONLFilename() string { - _, relPath := checkGitForIssues() - if relPath == "" { - return "" - } - return filepath.Base(relPath) -} -``` - -**Alternative Simpler Approach for Fix B**: -Instead of making `findJSONLPath()` git-aware, ensure import immediately exports to local file: -```go -// In cmd/bd/init.go after successful importFromGit (line 148): -if err := importFromGit(ctx, initDBPath, store, jsonlPath); err != nil { - // ...error handling... -} else { - // CRITICAL: Immediately export to local to prevent daemon race - localPath := filepath.Join(".beads", filepath.Base(jsonlPath)) - if err := exportToJSONL(ctx, store, localPath); err != nil { - fmt.Fprintf(os.Stderr, "Warning: failed to export after import: %v\n", err) - } - fmt.Fprintf(os.Stderr, "✓ Successfully imported %d issues from git.\n\n", issueCount) -} -``` - -**Race Condition Warning**: -After `rm -rf .beads/`, there's a timing window: -1. `bd init` runs, imports from git's `beads.jsonl` -2. Import schedules auto-flush (5-second debounce) -3. Daemon auto-starts before flush completes -4. Daemon calls `findJSONLPath()` → no local file yet → creates wrong `issues.jsonl` - -**Solution**: Import must **immediately create local JSONL** (no debounce) to win the race. - -**Revised Priority**: -- Fix A: P0 - Blocks everything, enables git detection -- Fix C: P0 - Prevents silent failures, critical safety net -- Fix B: P0 - Prevents wrong file creation (OR immediate export) -- Fix D: P1 - Nice recovery but redundant if A+B+C work - -### Precedence Rules (All Fixes) - -**When checking git HEAD**: -1. First try `.beads/beads.jsonl` -2. Then try `.beads/issues.jsonl` -3. Ignore non-canonical names (archive.jsonl, backup.jsonl, etc.) - -**When multiple local JSONL files exist**: -- Use existing `beads.FindJSONLPath()` glob behavior (first match) -- This preserves backward compatibility - -### Long Term (P2) -1. **Unified JSONL naming** - - Standardize on one filename (recommend `beads.jsonl`) - - Migration path for old `issues.jsonl` - - Update all code paths consistently - - Optional: Store chosen JSONL filename in DB metadata - -2. **Git-aware init** ✅ PARTIALLY DONE - - `bd init` should be git-aware ✅ EXISTS (commit 7f82708) - - Detect tracked JSONL and import automatically ❌ BROKEN (wrong filename) - - Make this the default happy path ✅ WILL BE FIXED by Fix A - -## Implementation Plan (Epic Structure) - -**Epic**: Fix database reinitialization data loss bug - -**Child Issues** (in dependency order): -1. **Fix A**: checkGitForIssues() filename detection (P0, <1h) - - Update autoimport.go:70-96 to try beads.jsonl then issues.jsonl - - Test: verify detects both filenames in git - - Blocks: Fix C (needs working detection) - -2. **Fix B-Alt**: Immediate export after import (P0, <1h) - - In init.go after importFromGit(), immediately call exportToJSONL() - - Prevents daemon race condition - - Simpler than making findJSONLPath() git-aware - - Test: verify local JSONL created with correct filename - -3. **Fix C**: Init safety check (P0, <1h) - - Add post-init verification in init.go - - Error and exit if DB empty but git has issues - - Depends: Fix A (uses checkGitForIssues) - - Test: verify fails loudly when import fails - -4. **Fix D**: Daemon startup import (P1, <1h) - - Add empty-DB check on daemon startup - - Auto-import if git has issues - - Depends: Fix A (uses checkGitForIssues) - - Test: verify daemon recovers from empty DB - -5. **Integration tests** (P0, 1-2h) - - Test fresh clone scenario - - Test `rm -rf .beads/` scenario - - Test daemon race condition (start daemon immediately after init) - - Test both beads.jsonl and issues.jsonl in git - -**Estimated Total**: 5-7 hours - -## Related Issues - -- **bd-112**: Warn when multiple beads databases detected (filed in ~/src/beads) -- **GH #142**: External_ref import feature (not directly related but shows import complexity) -- Commit d1d3fcd: Renamed `issues.jsonl → beads.jsonl` -- Commit 7f82708: "Fix bd init to auto-import issues from git on fresh clone" - -## Test Cases Needed - -1. **Fresh Clone Scenario** - ```bash - git clone repo - cd repo - bd init - # Should auto-import from .beads/beads.jsonl - # Should create local .beads/beads.jsonl immediately - bd stats --json | jq '.total_issues' # Should match git count - ``` - -2. **Database Removal Scenario (Primary Bug)** - ```bash - rm -rf .beads/ - bd init - # Should detect git-tracked JSONL and import - bd stats --json | jq '.total_issues' # Should be >0, not 0 - ls .beads/*.jsonl # Should be beads.jsonl, NOT issues.jsonl - ``` - -3. **Race Condition Scenario (Daemon Startup)** - ```bash - rm -rf .beads/ - bd init & # Start init in background - sleep 0.1 - bd ready # Triggers daemon auto-start - wait - # Daemon should NOT create issues.jsonl - # Should use beads.jsonl from git - ls .beads/*.jsonl - ``` - -4. **Legacy Filename Support (issues.jsonl)** - ```bash - # Git has .beads/issues.jsonl (not beads.jsonl) - rm -rf .beads/ - bd init - # Should still import correctly - ls .beads/*.jsonl # Should be issues.jsonl (matches git) - ``` - -5. **Multiple Workspace Scenario** - ```bash - # Two separate clones - ~/src/beads/ # database 1 - ~/src/fred/beads/ # database 2 - # Each should maintain separate state correctly - # Each should use correct JSONL filename from its own git - ``` - -6. **Daemon Restart Scenario** - ```bash - bd daemon --stop - rm .beads/bd.db - bd daemon # auto-start - # Should import from git on startup - bd stats --json | jq '.total_issues' # Should be >0 - ``` - -7. **Init Safety Check Scenario** - ```bash - # Simulate import failure - rm -rf .beads/ - chmod 000 .beads # Prevent creation - bd init 2>&1 | grep ERROR - # Should fail with clear error, not silent success - ``` - -## Root Cause Analysis - CONFIRMED - -### Primary Bug: Hardcoded Filename in checkGitForIssues() - -**File**: `cmd/bd/autoimport.go:76` -**Problem**: Hardcoded to `"issues.jsonl"` but git tracks `"beads.jsonl"` - -```go -// Line 76 - HARDCODED FILENAME -relPath, err := filepath.Rel(gitRoot, filepath.Join(beadsDir, "issues.jsonl")) -``` - -### Secondary Bug: Daemon Creates Wrong JSONL File - -**File**: `cmd/bd/main.go:findJSONLPath()`, `beads.go:FindJSONLPath()` -**Problem**: When no local JSONL exists, defaults to `"issues.jsonl"` without checking git HEAD - -**Code Flow**: -1. `FindJSONLPath()` globs for `*.jsonl` in `.beads/` (line 137) -2. If none found, defaults to `"issues.jsonl"` (line 144) -3. Daemon exports to empty `issues.jsonl`, ignoring `beads.jsonl` in git - -### Why Auto-Import Failed - -1. **bd init** called `checkGitForIssues()` → looked for `HEAD:.beads/issues.jsonl` -2. Git only has `HEAD:.beads/beads.jsonl` → check returned 0 issues -3. No import triggered, DB stayed empty -4. Daemon started, called `findJSONLPath()` → found no local JSONL -5. Defaulted to `issues.jsonl`, exported 0 issues to empty file -6. **Silent data loss complete** - -## Questions for Investigation - -1. ✅ Why did auto-import not trigger after `bd init`? - - **ANSWERED**: checkGitForIssues() hardcoded to issues.jsonl, git has beads.jsonl -2. ✅ Is there auto-import code that's not being called? - - **ANSWERED**: Auto-import code ran but found 0 issues due to wrong filename -3. ✅ When should daemon vs CLI handle import? - - **ANSWERED**: Both should handle; daemon on startup if DB empty + git has JSONL -4. ✅ Should we enforce single JSONL filename across codebase? - - **ANSWERED**: Support both with precedence: beads.jsonl > issues.jsonl -5. ✅ How do we prevent this silent data loss in future? - - **ANSWERED**: See proposed fixes below - -## Severity Justification: P0 - -This is a **critical data loss bug**: -- ✅ Silent failure (no error, no warning) -- ✅ Complete data loss (0 issues after 202) -- ✅ Core workflow broken (init + auto-import) -- ✅ Multi-workspace scenarios broken -- ✅ User cannot recover without manual intervention -- ✅ Breaks trust in beads reliability - -**Recommendation**: Investigate and fix immediately before 1.0 release.