diff --git a/.beads/bd.jsonl b/.beads/bd.jsonl index 761cc29d..97785984 100644 --- a/.beads/bd.jsonl +++ b/.beads/bd.jsonl @@ -68,7 +68,7 @@ {"id":"bd-16","title":"Add lifecycle safety docs and tests for UnderlyingDB() method","description":"The new UnderlyingDB() method exposes the raw *sql.DB connection for extensions like VC to create their own tables. While database/sql is concurrency-safe, there are lifecycle and misuse risks that need documentation and testing.\n\n**What needs to be done:**\n\n1. **Enhanced documentation** - Expand UnderlyingDB() comments to warn:\n - Callers MUST NOT call Close() on returned DB\n - Do NOT change pool/driver settings (SetMaxOpenConns, SetConnMaxIdleTime)\n - Do NOT modify SQLite PRAGMAs (WAL mode, journal, etc.)\n - Expect errors after Storage.Close() - use contexts\n - Keep write transactions short to avoid blocking core storage\n\n2. **Add lifecycle tracking** - Implement closed flag:\n - Add atomic.Bool closed field to SQLiteStorage\n - Set flag in Close(), clear in New()\n - Optional: Add IsClosed() bool method\n\n3. **Add safety tests** (run with -race):\n - TestUnderlyingDB_ConcurrentAccess - N goroutines using UnderlyingDB() during normal storage ops\n - TestUnderlyingDB_AfterClose - Verify operations fail cleanly after storage closed\n - TestUnderlyingDB_CreateExtensionTables - Create VC table with FK to issues, verify FK enforcement\n - TestUnderlyingDB_LongTxDoesNotCorrupt - Ensure long read tx doesn't block writes indefinitely\n\n**Why this matters:**\nVC will use this to create tables in the same database. Need to ensure production-ready safety without over-engineering.\n\n**Estimated effort:** S+S+S = M total (1-3h)","design":"Oracle recommends \"simple path\": enhanced docs + minimal guardrails + focused tests. See oracle output for detailed rationale on concurrency safety, lifecycle risks, and when to consider advanced path (wrapping interface).","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-22T17:07:56.812983-07:00","updated_at":"2025-10-25T23:15:33.476053-07:00","closed_at":"2025-10-22T20:10:52.636372-07:00"} {"id":"bd-160","title":"Critical: Multi-clone sync is fundamentally broken","description":"## Problem\n\nTwo clones of the same beads repo working on non-overlapping issues cannot stay in sync. The JSONL export/import mechanism creates catastrophic divergence instead of keeping databases synchronized.\n\n## What Happened (2025-10-26)\n\nTwo repos working simultaneously:\n- ~/src/beads (bd.db, 159 issues) - worked on bd-153, bd-152, bd-150, closed them\n- ~/src/fred/beads (beads.db, 165 issues) - worked on bd-159, bd-160-164\n\nResult after attempting sync:\n- Databases completely diverged (159 vs 165 issues)\n- JSONL files contain conflicting state\n- Database corruption in fred/beads\n- bd-150/152/153 show as closed in one repo, open in the other\n- No clear recovery path without manual database copying\n- git pull + bd sync does NOT synchronize state\n\n## Root Cause Analysis\n\n### SMOKING GUN: Daemon Import is a NO-OP\n**Location**: cmd/bd/daemon.go:791-797\n\nThe daemon's importToJSONLWithStore() function returns nil without actually importing.\nThis means the daemon exports DB to JSONL, commits, pulls from remote, but NEVER imports remote changes back into the database.\n\nResult: Remote changes are pulled but never imported, daemon keeps exporting stale state.\n\n### Other Root Causes\n\n1. **Database naming inconsistency**: One repo uses bd.db, other uses beads.db - no enforcement\n2. **Daemon state divergence**: Each repo's daemon maintains separate state, never converges\n3. **JSONL import/export race conditions**: Auto-import can overwrite local changes before export\n4. **No conflict resolution**: When databases diverge, there's no merge strategy\n5. **Timestamp-only changes**: bd-159 - exports trigger even with no real changes\n6. **Multiple daemons**: No coordination between daemon instances\n\n## Impact\n\n**Beads is unusable for multi-developer or multi-agent workflows**. The core promise - git-based sync via JSONL - is broken.\n\n## Fix Strategy (Epic)\n\nThis issue is tracked as an EPIC with child issues:\n\n### Phase 1: Stop the Bleeding (P0)\n- Implement daemon JSONL import (fixes the NO-OP)\n- Add database integrity checks\n- Fix timestamp-only exports (bd-159)\n\n### Phase 2: Database Consistency (P0)\n- Enforce canonical database naming\n- Add database fingerprinting\n- Migration tooling\n\n### Phase 3: Conflict Resolution (P1)\n- Implement version tracking\n- Three-way merge detection\n- Interactive conflict resolution\n\n### Phase 4: Testing \u0026 Validation (P1)\n- Multi-clone integration tests\n- Stress tests\n- Documentation\n\n## Severity\n\nP0 - This breaks the fundamental use case of beads. Without reliable sync, the tool is unusable for any multi-agent or team scenario.","status":"open","priority":0,"issue_type":"bug","created_at":"2025-10-26T19:42:43.355244-07:00","updated_at":"2025-10-26T19:53:08.681645-07:00"} {"id":"bd-161","title":"Implement daemon JSONL import (fix NO-OP stub)","description":"## Critical Bug\n\nThe daemon's sync loop calls importToJSONLWithStore() but this function is a NO-OP stub that returns nil without importing any changes.\n\n## Location\n\n**File**: cmd/bd/daemon.go:791-797\n\nCurrent implementation:\n```go\nfunc importToJSONLWithStore(ctx context.Context, store storage.Storage, jsonlPath string) error {\n // TODO Phase 4: Implement direct import for daemon\n // Currently a no-op - daemon doesn't import git changes into DB\n return nil\n}\n```\n\n## Impact\n\nThis is the PRIMARY cause of bd-160. When the daemon:\n1. Exports DB → JSONL\n2. Commits changes\n3. Pulls from remote (gets other clone's changes)\n4. Calls importToJSONLWithStore() ← **Does nothing!**\n5. Pushes commits (overwrites remote with stale state)\n\nResult: Perpetual divergence between clones.\n\n## Implementation Approach\n\nReplace the NO-OP with actual import logic:\n\n```go\nfunc importToJSONLWithStore(ctx context.Context, store storage.Storage, jsonlPath string) error {\n // Read JSONL file\n file, err := os.Open(jsonlPath)\n if err != nil {\n return fmt.Errorf(\"failed to open JSONL: %w\", err)\n }\n defer file.Close()\n \n // Parse all issues\n var issues []*types.Issue\n scanner := bufio.NewScanner(file)\n for scanner.Scan() {\n var issue types.Issue\n if err := json.Unmarshal(scanner.Bytes(), \u0026issue); err != nil {\n return fmt.Errorf(\"failed to parse issue: %w\", err)\n }\n issues = append(issues, \u0026issue)\n }\n \n if err := scanner.Err(); err != nil {\n return fmt.Errorf(\"failed to read JSONL: %w\", err)\n }\n \n // Use existing import logic with auto-conflict resolution\n opts := ImportOptions{\n ResolveCollisions: true, // Auto-resolve ID conflicts\n DryRun: false,\n SkipUpdate: false,\n Strict: false,\n }\n \n _, err = importIssuesCore(ctx, \"\", store, issues, opts)\n return err\n}\n```\n\n## Testing\n\nAfter implementation, test with:\n```bash\n# Create two clones\ngit init repo1 \u0026\u0026 cd repo1 \u0026\u0026 bd init \u0026\u0026 bd daemon\nbd new \"Issue A\"\ngit add . \u0026\u0026 git commit -m \"init\"\n\ncd .. \u0026\u0026 git clone repo1 repo2 \u0026\u0026 cd repo2 \u0026\u0026 bd init \u0026\u0026 bd daemon\n\n# Make changes in repo1\ncd ../repo1 \u0026\u0026 bd new \"Issue B\"\n\n# Wait for daemon sync, then check repo2\nsleep 10\ncd ../repo2 \u0026\u0026 bd list # Should show both Issue A and B\n```\n\n## Success Criteria\n\n- Daemon imports remote changes after git pull\n- Issue count converges across clones within one sync cycle\n- No manual intervention needed\n- Existing collision resolution logic handles conflicts\n\n## Estimated Effort\n\n30-60 minutes\n\n## Priority\n\nP0 - This is the critical path fix for bd-160","status":"closed","priority":0,"issue_type":"task","created_at":"2025-10-26T19:53:55.313039-07:00","updated_at":"2025-10-26T20:04:41.902916-07:00","closed_at":"2025-10-26T20:04:41.902916-07:00","dependencies":[{"issue_id":"bd-161","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:53:55.3136-07:00","created_by":"daemon"}]} -{"id":"bd-162","title":"Add database integrity checks to sync operations","description":"## Problem\n\nWhen databases diverge (due to the import NO-OP bug or race conditions), there are no safety checks to detect or prevent catastrophic data loss.\n\nNeed integrity checks before/after sync operations to catch divergence early.\n\n## Implementation Locations\n\n**Pre-export checks** (cmd/bd/daemon.go:948, sync.go:108):\n- Before exportToJSONLWithStore()\n- Before exportToJSONL()\n\n**Post-import checks** (cmd/bd/daemon.go:985):\n- After importToJSONLWithStore()\n\n## Checks to Implement\n\n### 1. Database vs JSONL Count Divergence\n\nBefore export:\n```go\nfunc validatePreExport(store storage.Storage, jsonlPath string) error {\n dbIssues, _ := store.SearchIssues(ctx, \"\", types.IssueFilter{})\n dbCount := len(dbIssues)\n \n jsonlCount, _ := countIssuesInJSONL(jsonlPath)\n \n if dbCount == 0 \u0026\u0026 jsonlCount \u003e 0 {\n return fmt.Errorf(\"refusing to export empty DB over %d issues in JSONL\", jsonlCount)\n }\n \n divergencePercent := math.Abs(float64(dbCount-jsonlCount)) / float64(jsonlCount) * 100\n if divergencePercent \u003e 50 {\n log.Printf(\"WARNING: DB has %d issues, JSONL has %d (%.1f%% divergence)\", \n dbCount, jsonlCount, divergencePercent)\n log.Printf(\"This suggests sync failure - investigate before proceeding\")\n }\n \n return nil\n}\n```\n\n### 2. Duplicate ID Detection\n\n```go\nfunc checkDuplicateIDs(store storage.Storage) error {\n // Query for duplicate IDs\n rows, _ := db.Query(`\n SELECT id, COUNT(*) as cnt \n FROM issues \n GROUP BY id \n HAVING cnt \u003e 1\n `)\n \n var duplicates []string\n for rows.Next() {\n var id string\n var count int\n rows.Scan(\u0026id, \u0026count)\n duplicates = append(duplicates, fmt.Sprintf(\"%s (x%d)\", id, count))\n }\n \n if len(duplicates) \u003e 0 {\n return fmt.Errorf(\"database corruption: duplicate IDs: %v\", duplicates)\n }\n return nil\n}\n```\n\n### 3. Orphaned Dependencies\n\n```go\nfunc checkOrphanedDeps(store storage.Storage) ([]string, error) {\n // Find dependencies pointing to non-existent issues\n rows, _ := db.Query(`\n SELECT DISTINCT d.depends_on_id \n FROM dependencies d \n LEFT JOIN issues i ON d.depends_on_id = i.id \n WHERE i.id IS NULL\n `)\n \n var orphaned []string\n for rows.Next() {\n var id string\n rows.Scan(\u0026id)\n orphaned = append(orphaned, id)\n }\n \n if len(orphaned) \u003e 0 {\n log.Printf(\"WARNING: Found %d orphaned dependencies: %v\", len(orphaned), orphaned)\n }\n \n return orphaned, nil\n}\n```\n\n### 4. Post-Import Validation\n\nAfter import, verify:\n```go\nfunc validatePostImport(before, after int) error {\n if after \u003c before {\n return fmt.Errorf(\"import reduced issue count: %d → %d (data loss!)\", before, after)\n }\n if after == before {\n log.Printf(\"Import complete: no changes\")\n } else {\n log.Printf(\"Import complete: %d → %d issues (+%d)\", before, after, after-before)\n }\n return nil\n}\n```\n\n## Integration Points\n\nAdd to daemon sync loop (daemon.go:920-999):\n```go\n// Before export\nif err := validatePreExport(store, jsonlPath); err != nil {\n log.log(\"Pre-export validation failed: %v\", err)\n return\n}\n\n// Export...\n\n// Before import\nbeforeCount := countDBIssues(store)\n\n// Import...\n\n// After import\nafterCount := countDBIssues(store)\nif err := validatePostImport(beforeCount, afterCount); err != nil {\n log.log(\"Post-import validation failed: %v\", err)\n}\n```\n\n## Testing\n\nCreate test scenarios:\n1. Empty DB, non-empty JSONL → should error\n2. Duplicate IDs in DB → should error\n3. Orphaned dependencies → should warn\n4. Import reduces count → should error\n\n## Success Criteria\n\n- Catches divergence \u003e50% before export\n- Detects duplicate IDs\n- Reports orphaned dependencies\n- Validates import doesn't lose data\n- All checks logged clearly\n\n## Estimated Effort\n\n2-3 hours\n\n## Priority\n\nP0 - Safety checks prevent data loss during sync","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-26T19:54:22.558861-07:00","updated_at":"2025-10-26T19:54:22.558861-07:00","dependencies":[{"issue_id":"bd-162","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:54:22.55941-07:00","created_by":"daemon"}]} +{"id":"bd-162","title":"Add database integrity checks to sync operations","description":"## Problem\n\nWhen databases diverge (due to the import NO-OP bug or race conditions), there are no safety checks to detect or prevent catastrophic data loss.\n\nNeed integrity checks before/after sync operations to catch divergence early.\n\n## Implementation Locations\n\n**Pre-export checks** (cmd/bd/daemon.go:948, sync.go:108):\n- Before exportToJSONLWithStore()\n- Before exportToJSONL()\n\n**Post-import checks** (cmd/bd/daemon.go:985):\n- After importToJSONLWithStore()\n\n## Checks to Implement\n\n### 1. Database vs JSONL Count Divergence\n\nBefore export:\n```go\nfunc validatePreExport(store storage.Storage, jsonlPath string) error {\n dbIssues, _ := store.SearchIssues(ctx, \"\", types.IssueFilter{})\n dbCount := len(dbIssues)\n \n jsonlCount, _ := countIssuesInJSONL(jsonlPath)\n \n if dbCount == 0 \u0026\u0026 jsonlCount \u003e 0 {\n return fmt.Errorf(\"refusing to export empty DB over %d issues in JSONL\", jsonlCount)\n }\n \n divergencePercent := math.Abs(float64(dbCount-jsonlCount)) / float64(jsonlCount) * 100\n if divergencePercent \u003e 50 {\n log.Printf(\"WARNING: DB has %d issues, JSONL has %d (%.1f%% divergence)\", \n dbCount, jsonlCount, divergencePercent)\n log.Printf(\"This suggests sync failure - investigate before proceeding\")\n }\n \n return nil\n}\n```\n\n### 2. Duplicate ID Detection\n\n```go\nfunc checkDuplicateIDs(store storage.Storage) error {\n // Query for duplicate IDs\n rows, _ := db.Query(`\n SELECT id, COUNT(*) as cnt \n FROM issues \n GROUP BY id \n HAVING cnt \u003e 1\n `)\n \n var duplicates []string\n for rows.Next() {\n var id string\n var count int\n rows.Scan(\u0026id, \u0026count)\n duplicates = append(duplicates, fmt.Sprintf(\"%s (x%d)\", id, count))\n }\n \n if len(duplicates) \u003e 0 {\n return fmt.Errorf(\"database corruption: duplicate IDs: %v\", duplicates)\n }\n return nil\n}\n```\n\n### 3. Orphaned Dependencies\n\n```go\nfunc checkOrphanedDeps(store storage.Storage) ([]string, error) {\n // Find dependencies pointing to non-existent issues\n rows, _ := db.Query(`\n SELECT DISTINCT d.depends_on_id \n FROM dependencies d \n LEFT JOIN issues i ON d.depends_on_id = i.id \n WHERE i.id IS NULL\n `)\n \n var orphaned []string\n for rows.Next() {\n var id string\n rows.Scan(\u0026id)\n orphaned = append(orphaned, id)\n }\n \n if len(orphaned) \u003e 0 {\n log.Printf(\"WARNING: Found %d orphaned dependencies: %v\", len(orphaned), orphaned)\n }\n \n return orphaned, nil\n}\n```\n\n### 4. Post-Import Validation\n\nAfter import, verify:\n```go\nfunc validatePostImport(before, after int) error {\n if after \u003c before {\n return fmt.Errorf(\"import reduced issue count: %d → %d (data loss!)\", before, after)\n }\n if after == before {\n log.Printf(\"Import complete: no changes\")\n } else {\n log.Printf(\"Import complete: %d → %d issues (+%d)\", before, after, after-before)\n }\n return nil\n}\n```\n\n## Integration Points\n\nAdd to daemon sync loop (daemon.go:920-999):\n```go\n// Before export\nif err := validatePreExport(store, jsonlPath); err != nil {\n log.log(\"Pre-export validation failed: %v\", err)\n return\n}\n\n// Export...\n\n// Before import\nbeforeCount := countDBIssues(store)\n\n// Import...\n\n// After import\nafterCount := countDBIssues(store)\nif err := validatePostImport(beforeCount, afterCount); err != nil {\n log.log(\"Post-import validation failed: %v\", err)\n}\n```\n\n## Testing\n\nCreate test scenarios:\n1. Empty DB, non-empty JSONL → should error\n2. Duplicate IDs in DB → should error\n3. Orphaned dependencies → should warn\n4. Import reduces count → should error\n\n## Success Criteria\n\n- Catches divergence \u003e50% before export\n- Detects duplicate IDs\n- Reports orphaned dependencies\n- Validates import doesn't lose data\n- All checks logged clearly\n\n## Estimated Effort\n\n2-3 hours\n\n## Priority\n\nP0 - Safety checks prevent data loss during sync","status":"closed","priority":0,"issue_type":"task","created_at":"2025-10-26T19:54:22.558861-07:00","updated_at":"2025-10-26T20:17:37.981054-07:00","closed_at":"2025-10-26T20:17:37.981054-07:00","dependencies":[{"issue_id":"bd-162","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:54:22.55941-07:00","created_by":"daemon"}]} {"id":"bd-164","title":"Fix timestamp-only export deduplication (bd-159)","description":"## Problem\n\nExport deduplication logic is supposed to skip timestamp-only changes, but it's not working. This causes:\n- Spurious git commits every sync cycle\n- Increased race condition window\n- Harder to detect real changes\n- Amplifies bd-160 sync issues\n\nRelated to bd-159.\n\n## Location\n\n**File**: cmd/bd/export.go:236-246\n\nCurrent code clears dirty flags for all exported issues:\n```go\nif output == \"\" || output == findJSONLPath() {\n if err := store.ClearDirtyIssuesByID(ctx, exportedIDs); err != nil {\n fmt.Fprintf(os.Stderr, \"Warning: failed to clear dirty issues: %v\\n\", err)\n }\n clearAutoFlushState()\n}\n```\n\nProblem: No check whether issue actually changed (beyond timestamps).\n\n## Root Cause\n\nIssues are marked dirty on ANY update, including:\n- Timestamp updates (UpdatedAt field)\n- No-op updates (same values written)\n- Database reopens (sqlite WAL journal replays)\n\n## Implementation Approach\n\n### 1. Add Content Hash to dirty_issues Table\n\n```sql\nALTER TABLE dirty_issues ADD COLUMN content_hash TEXT;\n```\n\nThe hash should exclude timestamp fields:\n```go\nfunc computeIssueContentHash(issue *types.Issue) string {\n // Clone issue and zero out timestamps\n normalized := *issue\n normalized.CreatedAt = time.Time{}\n normalized.UpdatedAt = time.Time{}\n \n // Serialize to JSON\n data, _ := json.Marshal(normalized)\n \n // SHA256 hash\n hash := sha256.Sum256(data)\n return hex.EncodeToString(hash[:])\n}\n```\n\n### 2. Track Previous Export State\n\nStore issue snapshots in issue_snapshots table (already exists):\n```go\nfunc saveExportSnapshot(ctx context.Context, store storage.Storage, issue *types.Issue) error {\n snapshot := \u0026types.IssueSnapshot{\n IssueID: issue.ID,\n SnapshotAt: time.Now(),\n Title: issue.Title,\n Description: issue.Description,\n Status: issue.Status,\n // ... all fields except timestamps\n }\n return store.SaveSnapshot(ctx, snapshot)\n}\n```\n\n### 3. Deduplicate During Export\n\nIn export.go:\n```go\n// Before encoding each issue\nif shouldSkipExport(ctx, store, issue) {\n skippedCount++\n continue\n}\n\nfunc shouldSkipExport(ctx context.Context, store storage.Storage, issue *types.Issue) bool {\n // Get last exported snapshot\n snapshot, err := store.GetLatestSnapshot(ctx, issue.ID)\n if err != nil || snapshot == nil {\n return false // No snapshot, must export\n }\n \n // Compare content hash\n currentHash := computeIssueContentHash(issue)\n snapshotHash := computeSnapshotHash(snapshot)\n \n if currentHash == snapshotHash {\n // Timestamp-only change, skip\n log.Printf(\"Skipping %s (timestamp-only change)\", issue.ID)\n return true\n }\n \n return false\n}\n```\n\n### 4. Update on Real Export\n\nOnly save snapshot when actually exporting:\n```go\nfor _, issue := range issues {\n if shouldSkipExport(ctx, store, issue) {\n continue\n }\n \n if err := encoder.Encode(issue); err != nil {\n return err\n }\n \n // Save snapshot of exported state\n saveExportSnapshot(ctx, store, issue)\n exportedIDs = append(exportedIDs, issue.ID)\n}\n```\n\n## Alternative: Simpler Approach\n\nIf snapshot complexity is too much, use a simpler hash:\n\n```go\n// In dirty_issues table, store hash when marking dirty\nfunc markIssueDirty(ctx context.Context, issueID string, issue *types.Issue) error {\n hash := computeIssueContentHash(issue)\n \n _, err := db.Exec(`\n INSERT INTO dirty_issues (issue_id, content_hash) \n VALUES (?, ?)\n ON CONFLICT(issue_id) DO UPDATE SET content_hash = ?\n `, issueID, hash, hash)\n \n return err\n}\n\n// During export, check if hash changed\nfunc hasRealChanges(ctx context.Context, store storage.Storage, issue *types.Issue) bool {\n var storedHash string\n err := db.QueryRow(\"SELECT content_hash FROM dirty_issues WHERE issue_id = ?\", issue.ID).Scan(\u0026storedHash)\n if err != nil {\n return true // No stored hash, export it\n }\n \n currentHash := computeIssueContentHash(issue)\n return currentHash != storedHash\n}\n```\n\n## Testing\n\nTest cases:\n1. Update issue timestamp only → no export\n2. Update issue title → export\n3. Multiple timestamp updates → single export\n4. Database reopen → no spurious exports\n\nValidation:\n```bash\n# Start daemon, wait 1 hour\nbd daemon --interval 5s\nsleep 3600\n\n# Check git log - should be 0 commits\ngit log --since=\"1 hour ago\" --oneline | wc -l # expect: 0\n```\n\n## Success Criteria\n\n- Zero spurious exports for timestamp-only changes\n- Real changes still exported immediately\n- No performance regression\n- bd-159 resolved\n\n## Estimated Effort\n\n2-3 hours\n\n## Priority\n\nP0 - Prevents noise that amplifies bd-160 sync issues","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-26T19:54:58.248715-07:00","updated_at":"2025-10-26T19:54:58.248715-07:00","dependencies":[{"issue_id":"bd-164","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:54:58.24935-07:00","created_by":"daemon"},{"issue_id":"bd-164","depends_on_id":"bd-159","type":"related","created_at":"2025-10-26T19:54:58.249718-07:00","created_by":"daemon"}]} {"id":"bd-165","title":"Enforce canonical database naming (beads.db)","description":"## Problem\n\nCurrently, different clones can use different database filenames (bd.db, beads.db, issues.db), causing incompatibility when attempting to sync.\n\nExample from bd-160:\n- ~/src/beads uses bd.db\n- ~/src/fred/beads uses beads.db\n- Sync fails because they're fundamentally different databases\n\n## Solution\n\nEnforce a single canonical database name: **beads.db**\n\n## Implementation\n\n### 1. Define Canonical Name\n\n**File**: beads.go or constants.go (create if needed)\n\n```go\npackage beads\n\n// CanonicalDatabaseName is the required database filename for all beads repositories\nconst CanonicalDatabaseName = \"beads.db\"\n\n// LegacyDatabaseNames are old names that should be migrated\nvar LegacyDatabaseNames = []string{\"bd.db\", \"issues.db\", \"bugs.db\"}\n```\n\n### 2. Update bd init Command\n\n**File**: cmd/bd/init.go\n\n```go\nfunc runInit(cmd *cobra.Command, args []string) error {\n beadsDir := \".beads\"\n os.MkdirAll(beadsDir, 0755)\n \n dbPath := filepath.Join(beadsDir, beads.CanonicalDatabaseName)\n \n // Check for legacy databases\n for _, legacy := range beads.LegacyDatabaseNames {\n legacyPath := filepath.Join(beadsDir, legacy)\n if exists(legacyPath) {\n fmt.Printf(\"Found legacy database: %s\\n\", legacy)\n fmt.Printf(\"Migrating to canonical name: %s\\n\", beads.CanonicalDatabaseName)\n \n // Rename to canonical\n if err := os.Rename(legacyPath, dbPath); err != nil {\n return fmt.Errorf(\"migration failed: %w\", err)\n }\n fmt.Printf(\"✓ Migrated %s → %s\\n\", legacy, beads.CanonicalDatabaseName)\n }\n }\n \n // Create new database if doesn't exist\n if !exists(dbPath) {\n store, err := sqlite.New(dbPath)\n if err != nil {\n return err\n }\n defer store.Close()\n \n // Initialize with version metadata\n store.SetMetadata(context.Background(), \"bd_version\", Version)\n store.SetMetadata(context.Background(), \"db_name\", beads.CanonicalDatabaseName)\n }\n \n // ... rest of init\n}\n```\n\n### 3. Validate on Daemon Start\n\n**File**: cmd/bd/daemon.go:1076-1095\n\nUpdate the existing multiple-DB check:\n```go\n// Check for multiple .db files\nbeadsDir := filepath.Dir(daemonDBPath)\nmatches, err := filepath.Glob(filepath.Join(beadsDir, \"*.db\"))\nif err == nil \u0026\u0026 len(matches) \u003e 1 {\n log.log(\"Error: Multiple database files found:\")\n for _, match := range matches {\n log.log(\" - %s\", filepath.Base(match))\n }\n log.log(\"\")\n log.log(\"Beads requires a single canonical database: %s\", beads.CanonicalDatabaseName)\n log.log(\"Run 'bd init' to migrate legacy databases\")\n os.Exit(1)\n}\n\n// Validate using canonical name\nif filepath.Base(daemonDBPath) != beads.CanonicalDatabaseName {\n log.log(\"Error: Non-canonical database name: %s\", filepath.Base(daemonDBPath))\n log.log(\"Expected: %s\", beads.CanonicalDatabaseName)\n log.log(\"Run 'bd init' to migrate to canonical name\")\n os.Exit(1)\n}\n```\n\n### 4. Add Migration Command\n\n**File**: cmd/bd/migrate.go (create new)\n\n```go\nvar migrateCmd = \u0026cobra.Command{\n Use: \"migrate\",\n Short: \"Migrate database to canonical naming and schema\",\n Run: func(cmd *cobra.Command, args []string) {\n beadsDir := \".beads\"\n \n // Find current database\n var currentDB string\n for _, name := range append(beads.LegacyDatabaseNames, beads.CanonicalDatabaseName) {\n path := filepath.Join(beadsDir, name)\n if exists(path) {\n currentDB = path\n break\n }\n }\n \n if currentDB == \"\" {\n fmt.Println(\"No database found\")\n return\n }\n \n targetPath := filepath.Join(beadsDir, beads.CanonicalDatabaseName)\n \n if currentDB == targetPath {\n fmt.Println(\"Database already using canonical name\")\n return\n }\n \n // Backup first\n backupPath := currentDB + \".backup\"\n copyFile(currentDB, backupPath)\n fmt.Printf(\"Created backup: %s\\n\", backupPath)\n \n // Rename\n if err := os.Rename(currentDB, targetPath); err != nil {\n fmt.Fprintf(os.Stderr, \"Migration failed: %v\\n\", err)\n os.Exit(1)\n }\n \n fmt.Printf(\"✓ Migrated: %s → %s\\n\", filepath.Base(currentDB), beads.CanonicalDatabaseName)\n \n // Update metadata\n store, _ := sqlite.New(targetPath)\n defer store.Close()\n store.SetMetadata(context.Background(), \"db_name\", beads.CanonicalDatabaseName)\n },\n}\n```\n\n### 5. Update FindDatabasePath\n\n**File**: beads.go (or wherever FindDatabasePath is defined)\n\n```go\nfunc FindDatabasePath() string {\n beadsDir := findBeadsDir()\n if beadsDir == \"\" {\n return \"\"\n }\n \n // First try canonical name\n canonical := filepath.Join(beadsDir, CanonicalDatabaseName)\n if exists(canonical) {\n return canonical\n }\n \n // Check for legacy names (warn user)\n for _, legacy := range LegacyDatabaseNames {\n path := filepath.Join(beadsDir, legacy)\n if exists(path) {\n fmt.Fprintf(os.Stderr, \"WARNING: Using legacy database name: %s\\n\", legacy)\n fmt.Fprintf(os.Stderr, \"Run 'bd migrate' to upgrade to canonical name: %s\\n\", CanonicalDatabaseName)\n return path\n }\n }\n \n return \"\"\n}\n```\n\n## Testing\n\n```bash\n# Test migration\nmkdir test-repo \u0026\u0026 cd test-repo \u0026\u0026 git init\nmkdir .beads\nsqlite3 .beads/bd.db \"CREATE TABLE test (id int);\"\n\nbd init # Should detect and migrate bd.db → beads.db\n\n# Verify\nls .beads/*.db # Should only show beads.db\n\n# Test daemon rejection\nsqlite3 .beads/old.db \"CREATE TABLE test (id int);\"\nbd daemon # Should error: multiple databases found\n\n# Test clean init\nrm -rf test-repo2 \u0026\u0026 mkdir test-repo2 \u0026\u0026 cd test-repo2\nbd init # Should create .beads/beads.db directly\n```\n\n## Rollout Strategy\n\n1. Add migration logic to bd init\n2. Update FindDatabasePath to warn on legacy names\n3. Add 'bd migrate' command for manual migration\n4. Update docs to specify canonical name\n5. Add daemon validation after 2 releases\n\n## Success Criteria\n\n- All new repositories use beads.db\n- bd init auto-migrates legacy names\n- bd daemon rejects non-canonical names\n- Clear migration path for existing users\n- No data loss during migration\n\n## Estimated Effort\n\n3-4 hours\n\n## Priority\n\nP0 - Critical for multi-clone compatibility","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-26T19:55:39.056716-07:00","updated_at":"2025-10-26T19:55:39.056716-07:00","dependencies":[{"issue_id":"bd-165","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:55:39.057336-07:00","created_by":"daemon"}]} {"id":"bd-166","title":"Add database fingerprinting and validation","description":"## Problem\n\nWhen multiple clones exist, there's no validation that they're actually clones of the same repository. Different repos can accidentally share databases, causing data corruption.\n\nNeed database fingerprinting to ensure clones belong to the same logical repository.\n\n## Solution\n\nAdd repository fingerprint to database metadata and validate on daemon start.\n\n## Implementation\n\n### 1. Compute Repository ID\n\n**File**: pkg/fingerprint.go (create new)\n\n```go\npackage beads\n\nimport (\n \"crypto/sha256\"\n \"encoding/hex\"\n \"fmt\"\n \"os/exec\"\n)\n\n// ComputeRepoID generates a unique identifier for this git repository\nfunc ComputeRepoID() (string, error) {\n // Get git remote URL (canonical repo identifier)\n cmd := exec.Command(\"git\", \"config\", \"--get\", \"remote.origin.url\")\n output, err := cmd.Output()\n if err != nil {\n // No remote configured, use local path\n cmd = exec.Command(\"git\", \"rev-parse\", \"--show-toplevel\")\n output, err = cmd.Output()\n if err != nil {\n return \"\", fmt.Errorf(\"not a git repository\")\n }\n }\n \n repoURL := strings.TrimSpace(string(output))\n \n // Normalize URL (remove .git suffix, https vs git@, etc.)\n repoURL = normalizeGitURL(repoURL)\n \n // SHA256 hash for privacy (don't expose repo URL in database)\n hash := sha256.Sum256([]byte(repoURL))\n return hex.EncodeToString(hash[:16]), nil // Use first 16 bytes\n}\n\nfunc normalizeGitURL(url string) string {\n // Convert git@github.com:user/repo.git → github.com/user/repo\n // Convert https://github.com/user/repo.git → github.com/user/repo\n url = strings.TrimSuffix(url, \".git\")\n url = strings.ReplaceAll(url, \"git@\", \"\")\n url = strings.ReplaceAll(url, \"https://\", \"\")\n url = strings.ReplaceAll(url, \"http://\", \"\")\n url = strings.ReplaceAll(url, \":\", \"/\")\n return url\n}\n\n// GetCloneID generates a unique ID for this specific clone (not shared with other clones)\nfunc GetCloneID() string {\n // Use hostname + path for uniqueness\n hostname, _ := os.Hostname()\n path, _ := os.Getwd()\n hash := sha256.Sum256([]byte(hostname + \":\" + path))\n return hex.EncodeToString(hash[:8])\n}\n```\n\n### 2. Store Fingerprint on Init\n\n**File**: cmd/bd/init.go\n\n```go\nfunc runInit(cmd *cobra.Command, args []string) error {\n // ... create database ...\n \n // Compute and store repo ID\n repoID, err := beads.ComputeRepoID()\n if err != nil {\n fmt.Fprintf(os.Stderr, \"Warning: could not compute repo ID: %v\\n\", err)\n } else {\n if err := store.SetMetadata(ctx, \"repo_id\", repoID); err != nil {\n return fmt.Errorf(\"failed to set repo_id: %w\", err)\n }\n fmt.Printf(\"Repository ID: %s\\n\", repoID[:8])\n }\n \n // Store clone ID\n cloneID := beads.GetCloneID()\n if err := store.SetMetadata(ctx, \"clone_id\", cloneID); err != nil {\n return fmt.Errorf(\"failed to set clone_id: %w\", err)\n }\n fmt.Printf(\"Clone ID: %s\\n\", cloneID)\n \n // Store creation timestamp\n if err := store.SetMetadata(ctx, \"created_at\", time.Now().Format(time.RFC3339)); err != nil {\n return fmt.Errorf(\"failed to set created_at: %w\", err)\n }\n \n return nil\n}\n```\n\n### 3. Validate on Database Open\n\n**File**: cmd/bd/daemon.go (in runDaemonLoop)\n\n```go\nfunc validateDatabaseFingerprint(store storage.Storage) error {\n ctx := context.Background()\n \n // Get stored repo ID\n storedRepoID, err := store.GetMetadata(ctx, \"repo_id\")\n if err != nil \u0026\u0026 err.Error() != \"metadata key not found: repo_id\" {\n return fmt.Errorf(\"failed to read repo_id: %w\", err)\n }\n \n // If no repo_id, this is a legacy database - set it now\n if storedRepoID == \"\" {\n repoID, err := beads.ComputeRepoID()\n if err != nil {\n log.log(\"Warning: could not compute repo ID: %v\", err)\n return nil // Non-fatal for backward compat\n }\n \n log.log(\"Legacy database detected, setting repo_id: %s\", repoID[:8])\n if err := store.SetMetadata(ctx, \"repo_id\", repoID); err != nil {\n return fmt.Errorf(\"failed to set repo_id: %w\", err)\n }\n return nil\n }\n \n // Validate repo ID matches\n currentRepoID, err := beads.ComputeRepoID()\n if err != nil {\n log.log(\"Warning: could not compute current repo ID: %v\", err)\n return nil // Non-fatal\n }\n \n if storedRepoID != currentRepoID {\n return fmt.Errorf(`\nDATABASE MISMATCH DETECTED!\n\nThis database belongs to a different repository:\n Database repo ID: %s\n Current repo ID: %s\n\nThis usually means:\n 1. You copied a .beads directory from another repo (don't do this!)\n 2. Git remote URL changed (run 'bd migrate' to update)\n 3. Database corruption\n\nSolutions:\n - If remote URL changed: bd migrate --update-repo-id\n - If wrong database: rm -rf .beads \u0026\u0026 bd init\n - If correct database: BEADS_IGNORE_REPO_MISMATCH=1 bd daemon\n`, storedRepoID[:8], currentRepoID[:8])\n }\n \n return nil\n}\n\n// In runDaemonLoop, after opening database:\nif err := validateDatabaseFingerprint(store); err != nil {\n if os.Getenv(\"BEADS_IGNORE_REPO_MISMATCH\") != \"1\" {\n log.log(\"Error: %v\", err)\n os.Exit(1)\n }\n log.log(\"Warning: repo mismatch ignored (BEADS_IGNORE_REPO_MISMATCH=1)\")\n}\n```\n\n### 4. Add Update Command for Remote Changes\n\n**File**: cmd/bd/migrate.go\n\n```go\nvar updateRepoID bool\n\nfunc init() {\n migrateCmd.Flags().BoolVar(\u0026updateRepoID, \"update-repo-id\", false, \n \"Update repository ID (use after changing git remote)\")\n}\n\n// In migrate command:\nif updateRepoID {\n newRepoID, err := beads.ComputeRepoID()\n if err != nil {\n fmt.Fprintf(os.Stderr, \"Error: %v\\n\", err)\n os.Exit(1)\n }\n \n oldRepoID, _ := store.GetMetadata(ctx, \"repo_id\")\n \n fmt.Printf(\"Updating repository ID:\\n\")\n fmt.Printf(\" Old: %s\\n\", oldRepoID[:8])\n fmt.Printf(\" New: %s\\n\", newRepoID[:8])\n \n if err := store.SetMetadata(ctx, \"repo_id\", newRepoID); err != nil {\n fmt.Fprintf(os.Stderr, \"Error: %v\\n\", err)\n os.Exit(1)\n }\n \n fmt.Println(\"✓ Repository ID updated\")\n}\n```\n\n## Metadata Schema\n\nAdd to `metadata` table:\n\n| Key | Value | Description |\n|-----|-------|-------------|\n| repo_id | sha256(git_remote)[..16] | Repository fingerprint |\n| clone_id | sha256(hostname:path)[..8] | Clone-specific ID |\n| created_at | RFC3339 timestamp | Database creation time |\n| db_name | \"beads.db\" | Canonical database name |\n| bd_version | \"v0.x.x\" | Schema version |\n\n## Testing\n\n```bash\n# Test repo ID generation\ncd /tmp/test-repo \u0026\u0026 git init\ngit remote add origin https://github.com/user/repo.git\nbd init\nbd show-meta repo_id # Should show consistent hash\n\n# Test mismatch detection\ncd /tmp/other-repo \u0026\u0026 git init\ngit remote add origin https://github.com/other/repo.git\ncp -r /tmp/test-repo/.beads /tmp/other-repo/\nbd daemon # Should error: repo mismatch\n\n# Test migration\ngit remote set-url origin https://github.com/user/new-repo.git\nbd migrate --update-repo-id # Should update successfully\n```\n\n## Success Criteria\n\n- New databases automatically get repo_id\n- Daemon validates repo_id on start\n- Clear error messages on mismatch\n- Migration path for remote URL changes\n- Legacy databases automatically fingerprinted\n\n## Estimated Effort\n\n3-4 hours\n\n## Priority\n\nP0 - Prevents accidental database mixing across repos","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-26T19:56:18.53693-07:00","updated_at":"2025-10-26T19:56:18.53693-07:00","dependencies":[{"issue_id":"bd-166","depends_on_id":"bd-160","type":"blocks","created_at":"2025-10-26T19:56:18.537546-07:00","created_by":"daemon"}]}