diff --git a/.beads/beads.jsonl b/.beads/beads.jsonl index e7134f4d..c3d22bf9 100644 --- a/.beads/beads.jsonl +++ b/.beads/beads.jsonl @@ -31,7 +31,7 @@ {"id":"bd-1f4086c5","content_hash":"23fbff5ec79ea76cf9c60b64676ee446445c878e3cc17011b925a1ec167142c5","title":"Event-driven daemon architecture","description":"Replace 5-second polling sync loop with event-driven architecture that reacts instantly to changes. Eliminates stale data issues while reducing CPU ~60%. Key components: FileWatcher (fsnotify), Debouncer (500ms), RPC mutation events, optional git hooks. Target latency: \u003c500ms (vs 5000ms). See event_driven_daemon.md for full design.","notes":"Production-ready after 3 critical fixes (commit 349b892):\n- Skip redundant imports (mtime check prevents self-trigger loops)\n- Add server.Stop() in serverErrChan case (clean shutdown)\n- Fallback ticker (60s) when watcher unavailable (ensures remote sync)\n\nReady to make default after integration test (bd-1f4086c5.1) passes.","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-10-29T23:05:13.969484-07:00","updated_at":"2025-10-31T20:21:25.464736-07:00","closed_at":"2025-10-31T20:21:25.464736-07:00"} {"id":"bd-1f4086c5.1","content_hash":"ba5173c61613a29786641ba06a93427de87bed65ce39dbc3c3ddd2b6900f827e","title":"Integration test: mutation to export latency","description":"Measure time from bd create to JSONL update. Verify \u003c500ms latency. Test with multiple rapid mutations to verify batching.","notes":"Test added to daemon_test.go as TestMutationToExportLatency().\n\nCurrently skipped with note that it should be enabled once bd-146 (event-driven daemon) is fully implemented and enabled by default.\n\nThe test structure is complete:\n1. Sets up test environment with fast debounce (500ms)\n2. SingleMutationLatency: measures latency from mutation to JSONL update\n3. RapidMutationBatching: verifies multiple mutations batch into single export\n\nOnce event-driven mode is default, remove the t.Skip() line and the test will validate \u003c500ms latency.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-29T20:49:49.103759-07:00","updated_at":"2025-10-30T17:12:58.195867-07:00","closed_at":"2025-10-29T14:19:19.808139-07:00","dependencies":[{"issue_id":"bd-1f4086c5.1","depends_on_id":"bd-1f4086c5","type":"parent-child","created_at":"2025-10-29T20:49:49.107244-07:00","created_by":"import-remap"}]} {"id":"bd-22e0bde9","content_hash":"4c03fb79e67c0948d0d887b56fcbf71ed3b987e4bfd84628d7b9b2fa047a61fa","title":"Add TestNWayCollision for 5+ clones","description":"## Overview\nAdd comprehensive tests for N-way (5+) collision resolution to verify the solution scales beyond 3 clones.\n\n## Purpose\nWhile TestThreeCloneCollision validates the basic N-way case, we need to verify:\n1. Solution scales to arbitrary N\n2. Performance is acceptable with more clones\n3. Convergence time is bounded\n4. No edge cases in larger collision groups\n\n## Implementation Tasks\n\n### 1. Create TestFiveCloneCollision\nFile: beads_twoclone_test.go (or new beads_nway_test.go)\n\n```go\nfunc TestFiveCloneCollision(t *testing.T) {\n // Test with 5 clones creating same ID with different content\n // Verify all 5 clones converge after sync rounds\n \n t.Run(\"SequentialSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"A\", \"B\", \"C\", \"D\", \"E\")\n })\n \n t.Run(\"ReverseSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"E\", \"D\", \"C\", \"B\", \"A\")\n })\n \n t.Run(\"RandomSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"C\", \"A\", \"E\", \"B\", \"D\")\n })\n}\n```\n\n### 2. Implement generalized testNCloneCollision\nGeneralize the 3-clone test to handle arbitrary N:\n\n```go\nfunc testNCloneCollision(t *testing.T, numClones int, syncOrder ...string) {\n t.Helper()\n \n if len(syncOrder) != numClones {\n t.Fatalf(\"syncOrder length (%d) must match numClones (%d)\", \n len(syncOrder), numClones)\n }\n \n tmpDir := t.TempDir()\n \n // Setup remote and N clones\n remoteDir := setupBareRepo(t, tmpDir)\n cloneDirs := make(map[string]string)\n \n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n cloneDirs[name] = setupClone(t, tmpDir, remoteDir, name)\n }\n \n // Each clone creates issue with same ID but different content\n for name, dir := range cloneDirs {\n createIssue(t, dir, fmt.Sprintf(\"Issue from clone %s\", name))\n }\n \n // Sync in specified order\n for _, name := range syncOrder {\n syncClone(t, cloneDirs[name], name)\n }\n \n // Final pull for convergence\n for name, dir := range cloneDirs {\n finalPull(t, dir, name)\n }\n \n // Verify all clones have all N issues\n expectedTitles := make(map[string]bool)\n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n expectedTitles[fmt.Sprintf(\"Issue from clone %s\", name)] = true\n }\n \n for name, dir := range cloneDirs {\n titles := getTitles(t, dir)\n if !compareTitleSets(titles, expectedTitles) {\n t.Errorf(\"Clone %s missing issues: expected %v, got %v\", \n name, expectedTitles, titles)\n }\n }\n \n t.Log(\"✓ All\", numClones, \"clones converged successfully\")\n}\n```\n\n### 3. Add performance benchmarks\nTest convergence time and memory usage:\n\n```go\nfunc BenchmarkNWayCollision(b *testing.B) {\n for _, n := range []int{3, 5, 10, 20} {\n b.Run(fmt.Sprintf(\"N=%d\", n), func(b *testing.B) {\n for i := 0; i \u003c b.N; i++ {\n // Run N-way collision and measure time\n testNCloneCollisionBench(b, n)\n }\n })\n }\n}\n```\n\n### 4. Add convergence time tests\nVerify bounded convergence:\n\n```go\nfunc TestConvergenceTime(t *testing.T) {\n // Test that convergence happens within expected rounds\n // For N clones, should converge in at most N-1 sync rounds\n \n for n := 3; n \u003c= 10; n++ {\n t.Run(fmt.Sprintf(\"N=%d\", n), func(t *testing.T) {\n rounds := measureConvergenceRounds(t, n)\n maxExpected := n - 1\n if rounds \u003e maxExpected {\n t.Errorf(\"Convergence took %d rounds, expected ≤ %d\", \n rounds, maxExpected)\n }\n })\n }\n}\n```\n\n### 5. Add edge case tests\nTest boundary conditions:\n- All N clones have identical content (dedup works)\n- N-1 clones have same content, 1 differs\n- All N clones have unique content\n- Mix of collisions and non-collisions\n\n## Acceptance Criteria\n- TestFiveCloneCollision passes with all sync orders\n- All 5 clones converge to identical content\n- Performance is acceptable (\u003c 5 seconds for 5 clones)\n- Convergence time is bounded (≤ N-1 rounds)\n- Edge cases handled correctly\n- Benchmarks show scalability to 10+ clones\n\n## Files to Create/Modify\n- beads_twoclone_test.go or beads_nway_test.go\n- Add helper functions for N-clone setup\n\n## Testing Strategy\n\n### Test Matrix\n| N Clones | Sync Orders | Expected Result |\n|----------|-------------|-----------------|\n| 3 | A→B→C | Pass |\n| 3 | C→B→A | Pass |\n| 5 | A→B→C→D→E | Pass |\n| 5 | E→D→C→B→A | Pass |\n| 5 | Random | Pass |\n| 10 | Sequential | Pass |\n\n### Performance Targets\n- 3 clones: \u003c 2 seconds\n- 5 clones: \u003c 5 seconds\n- 10 clones: \u003c 15 seconds\n\n## Dependencies\n- Requires bd-cbed9619.5, bd-cbed9619.4, bd-cbed9619.3, bd-cbed9619.2 to be completed\n- TestThreeCloneCollision must pass first\n\n## Success Metrics\n- All tests pass for N ∈ {3, 5, 10}\n- Convergence time scales linearly (O(N))\n- Memory usage reasonable (\u003c 100MB for 10 clones)\n- No data corruption or loss in any scenario","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-29T23:05:13.974702-07:00","updated_at":"2025-10-31T12:00:43.197709-07:00","closed_at":"2025-10-31T12:00:43.197709-07:00"} -{"id":"bd-23a8","content_hash":"9d87b72ef47f258495d3a0d5ff4f1675400ec56324065da5b71902e1dbc8fe72","title":"Test simple issue","description":"","status":"open","priority":2,"issue_type":"task","created_at":"2025-11-02T17:11:04.464726-08:00","updated_at":"2025-11-02T17:11:04.464726-08:00"} +{"id":"bd-23a8","content_hash":"9d87b72ef47f258495d3a0d5ff4f1675400ec56324065da5b71902e1dbc8fe72","title":"Test simple issue","description":"","status":"open","priority":2,"issue_type":"task","created_at":"2025-11-02T17:11:04.464726-08:00","updated_at":"2025-11-02T17:11:04.464726-08:00","comments":[{"id":1,"issue_id":"bd-23a8","author":"stevey","text":"Testing the new bd comment alias!","created_at":"2025-11-03T02:41:01Z"},{"id":2,"issue_id":"bd-23a8","author":"stevey","text":"Another test with JSON output","created_at":"2025-11-03T02:41:01Z"},{"id":3,"issue_id":"bd-23a8","author":"stevey","text":"Test comment from file\n","created_at":"2025-11-03T02:41:01Z"}]} {"id":"bd-248bdc3e","content_hash":"8eaeb2dbef1ed6b25fc1bcf3bc5cd1b38a5cf5a487772558ba9fe12a149978f3","title":"Add optional post-merge git hook example for bd sync","description":"Create example git hook that auto-runs bd sync after git pull/merge.\n\nAdd to examples/git-hooks/:\n- post-merge hook that checks if .beads/issues.jsonl changed\n- If changed: run `bd sync` automatically\n- Make it optional/documented (not auto-installed)\n\nBenefits:\n- Zero-friction sync after git pull\n- Complements auto-detection as belt-and-suspenders\n\nNote: post-merge hook already exists for pre-commit/post-merge. Extend it to support sync.","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-25T22:47:14.668842-07:00","updated_at":"2025-10-30T17:12:58.218887-07:00"} {"id":"bd-2530","content_hash":"7056a386ee4802bce2b83a981aaac7858b5911938263d212f8f9d1f60bf2a706","title":"Issue with labels","description":"This is a description","design":"Use MVC pattern","acceptance_criteria":"All tests pass","status":"closed","priority":0,"issue_type":"feature","created_at":"2025-10-31T21:40:34.630173-07:00","updated_at":"2025-11-01T11:11:57.93151-07:00","closed_at":"2025-11-01T11:11:57.93151-07:00","labels":["bug","critical"]} {"id":"bd-2752a7a2","content_hash":"6b2a1aedbdbcb30b98d4a8196801953a1eb22204d63e31954ef9ab6020a7a26b","title":"Create cmd/bd/daemon_watcher.go (~150 LOC)","description":"Implement FileWatcher using fsnotify to watch JSONL file and git refs. Handle platform differences (inotify/FSEvents/ReadDirectoryChangesW). Include edge case handling for file rename, event storm, watcher failure.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-29T23:05:13.887269-07:00","updated_at":"2025-10-31T18:30:24.131535-07:00","closed_at":"2025-10-31T18:30:24.131535-07:00"} @@ -87,12 +87,12 @@ {"id":"bd-5aad5a9c","content_hash":"03aa8900fd0e6f4bd911364bb69d4446f239c6a5e9688f0fc4928f3de1259242","title":"Add TestNWayCollision for 5+ clones","description":"## Overview\nAdd comprehensive tests for N-way (5+) collision resolution to verify the solution scales beyond 3 clones.\n\n## Purpose\nWhile TestThreeCloneCollision validates the basic N-way case, we need to verify:\n1. Solution scales to arbitrary N\n2. Performance is acceptable with more clones\n3. Convergence time is bounded\n4. No edge cases in larger collision groups\n\n## Implementation Tasks\n\n### 1. Create TestFiveCloneCollision\nFile: beads_twoclone_test.go (or new beads_nway_test.go)\n\n```go\nfunc TestFiveCloneCollision(t *testing.T) {\n // Test with 5 clones creating same ID with different content\n // Verify all 5 clones converge after sync rounds\n \n t.Run(\"SequentialSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"A\", \"B\", \"C\", \"D\", \"E\")\n })\n \n t.Run(\"ReverseSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"E\", \"D\", \"C\", \"B\", \"A\")\n })\n \n t.Run(\"RandomSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"C\", \"A\", \"E\", \"B\", \"D\")\n })\n}\n```\n\n### 2. Implement generalized testNCloneCollision\nGeneralize the 3-clone test to handle arbitrary N:\n\n```go\nfunc testNCloneCollision(t *testing.T, numClones int, syncOrder ...string) {\n t.Helper()\n \n if len(syncOrder) != numClones {\n t.Fatalf(\"syncOrder length (%d) must match numClones (%d)\", \n len(syncOrder), numClones)\n }\n \n tmpDir := t.TempDir()\n \n // Setup remote and N clones\n remoteDir := setupBareRepo(t, tmpDir)\n cloneDirs := make(map[string]string)\n \n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n cloneDirs[name] = setupClone(t, tmpDir, remoteDir, name)\n }\n \n // Each clone creates issue with same ID but different content\n for name, dir := range cloneDirs {\n createIssue(t, dir, fmt.Sprintf(\"Issue from clone %s\", name))\n }\n \n // Sync in specified order\n for _, name := range syncOrder {\n syncClone(t, cloneDirs[name], name)\n }\n \n // Final pull for convergence\n for name, dir := range cloneDirs {\n finalPull(t, dir, name)\n }\n \n // Verify all clones have all N issues\n expectedTitles := make(map[string]bool)\n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n expectedTitles[fmt.Sprintf(\"Issue from clone %s\", name)] = true\n }\n \n for name, dir := range cloneDirs {\n titles := getTitles(t, dir)\n if !compareTitleSets(titles, expectedTitles) {\n t.Errorf(\"Clone %s missing issues: expected %v, got %v\", \n name, expectedTitles, titles)\n }\n }\n \n t.Log(\"✓ All\", numClones, \"clones converged successfully\")\n}\n```\n\n### 3. Add performance benchmarks\nTest convergence time and memory usage:\n\n```go\nfunc BenchmarkNWayCollision(b *testing.B) {\n for _, n := range []int{3, 5, 10, 20} {\n b.Run(fmt.Sprintf(\"N=%d\", n), func(b *testing.B) {\n for i := 0; i \u003c b.N; i++ {\n // Run N-way collision and measure time\n testNCloneCollisionBench(b, n)\n }\n })\n }\n}\n```\n\n### 4. Add convergence time tests\nVerify bounded convergence:\n\n```go\nfunc TestConvergenceTime(t *testing.T) {\n // Test that convergence happens within expected rounds\n // For N clones, should converge in at most N-1 sync rounds\n \n for n := 3; n \u003c= 10; n++ {\n t.Run(fmt.Sprintf(\"N=%d\", n), func(t *testing.T) {\n rounds := measureConvergenceRounds(t, n)\n maxExpected := n - 1\n if rounds \u003e maxExpected {\n t.Errorf(\"Convergence took %d rounds, expected ≤ %d\", \n rounds, maxExpected)\n }\n })\n }\n}\n```\n\n### 5. Add edge case tests\nTest boundary conditions:\n- All N clones have identical content (dedup works)\n- N-1 clones have same content, 1 differs\n- All N clones have unique content\n- Mix of collisions and non-collisions\n\n## Acceptance Criteria\n- TestFiveCloneCollision passes with all sync orders\n- All 5 clones converge to identical content\n- Performance is acceptable (\u003c 5 seconds for 5 clones)\n- Convergence time is bounded (≤ N-1 rounds)\n- Edge cases handled correctly\n- Benchmarks show scalability to 10+ clones\n\n## Files to Create/Modify\n- beads_twoclone_test.go or beads_nway_test.go\n- Add helper functions for N-clone setup\n\n## Testing Strategy\n\n### Test Matrix\n| N Clones | Sync Orders | Expected Result |\n|----------|-------------|-----------------|\n| 3 | A→B→C | Pass |\n| 3 | C→B→A | Pass |\n| 5 | A→B→C→D→E | Pass |\n| 5 | E→D→C→B→A | Pass |\n| 5 | Random | Pass |\n| 10 | Sequential | Pass |\n\n### Performance Targets\n- 3 clones: \u003c 2 seconds\n- 5 clones: \u003c 5 seconds\n- 10 clones: \u003c 15 seconds\n\n## Dependencies\n- Requires bd-cbed9619.5, bd-cbed9619.4, bd-cbed9619.3, bd-dcd6f14b to be completed\n- TestThreeCloneCollision must pass first\n\n## Success Metrics\n- All tests pass for N ∈ {3, 5, 10}\n- Convergence time scales linearly (O(N))\n- Memory usage reasonable (\u003c 100MB for 10 clones)\n- No data corruption or loss in any scenario","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-29T19:52:05.462747-07:00","updated_at":"2025-10-31T12:00:43.198413-07:00","closed_at":"2025-10-31T12:00:43.198413-07:00"} {"id":"bd-5b40a0bf","content_hash":"fd8a94f744b6504f677cc8e196c5a8d8b5d13b230200add5a8d9b7a54a537eb9","title":"Batch test 5","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-29T15:29:02.136118-07:00","updated_at":"2025-10-31T12:00:43.181513-07:00","closed_at":"2025-10-31T12:00:43.181513-07:00"} {"id":"bd-5b6e","content_hash":"f82a86b4aae21311f23c8511a242f16e96d03836300995fadd43b8bea945cefa","title":"Add tests for helper functions (GetDirtyIssueHash, GetAllDependencyRecords, export hashes)","description":"Several utility functions have 0% coverage:\n- GetDirtyIssueHash (dirty.go)\n- GetAllDependencyRecords (dependencies.go)\n- GetExportHash, SetExportHash, ClearAllExportHashes (hash.go)\n\nThese are lower priority but should have basic coverage.","status":"open","priority":4,"issue_type":"task","created_at":"2025-11-01T22:40:58.989976-07:00","updated_at":"2025-11-01T22:40:58.989976-07:00"} -{"id":"bd-5ce8","content_hash":"8371b74c06781a932c57c6b89a9803804d8e73e5b1abdaa5055d53197684457d","title":"Document protected branch workflow","description":"Create comprehensive documentation for protected branch workflow.\n\nTasks:\n- Add \"Protected Branch Workflow\" section to AGENTS.md\n- Create docs/PROTECTED_BRANCHES.md guide\n- Update README.md quick start\n- Add examples to examples/protected-branch/\n- Update bd init --help documentation\n- Add troubleshooting guide\n- Add migration guide for existing users\n- Record demo video (optional)\n\nEstimated effort: 2-3 days","acceptance_criteria":"- Clear quick start (\u003c 2 minutes to set up)\n- Detailed guide covers all scenarios\n- Examples work end-to-end\n- Troubleshooting answers common questions\n- Platform-agnostic (not GitHub-specific)","status":"open","priority":1,"issue_type":"task","created_at":"2025-11-02T15:22:35.59013-08:00","updated_at":"2025-11-02T17:19:13.185014-08:00","dependencies":[{"issue_id":"bd-5ce8","depends_on_id":"bd-a101","type":"parent-child","created_at":"2025-11-02T15:22:48.379767-08:00","created_by":"stevey"}]} +{"id":"bd-5ce8","content_hash":"23d02fea82e0a87bbb4c878472a368e093262e98c7d1f286374955a5b14ae1e5","title":"Document protected branch workflow","description":"Create comprehensive documentation for protected branch workflow.\n\nTasks:\n- Add \"Protected Branch Workflow\" section to AGENTS.md\n- Create docs/PROTECTED_BRANCHES.md guide\n- Update README.md quick start\n- Add examples to examples/protected-branch/\n- Update bd init --help documentation\n- Add troubleshooting guide\n- Add migration guide for existing users\n- Record demo video (optional)\n\nEstimated effort: 2-3 days","acceptance_criteria":"- Clear quick start (\u003c 2 minutes to set up)\n- Detailed guide covers all scenarios\n- Examples work end-to-end\n- Troubleshooting answers common questions\n- Platform-agnostic (not GitHub-specific)","notes":"Completed protected branch workflow documentation. Created comprehensive guide (docs/PROTECTED_BRANCHES.md), updated AGENTS.md with workflow section, added feature to README.md, and created working example (examples/protected-branch/). All commands verified working (bd init --branch, bd sync --status, bd sync --merge, bd config get/set sync.branch).","status":"closed","priority":1,"issue_type":"task","created_at":"2025-11-02T15:22:35.59013-08:00","updated_at":"2025-11-02T21:25:29.773166-08:00","closed_at":"2025-11-02T21:25:29.77317-08:00","dependencies":[{"issue_id":"bd-5ce8","depends_on_id":"bd-a101","type":"parent-child","created_at":"2025-11-02T15:22:48.379767-08:00","created_by":"stevey"}]} {"id":"bd-5dae5504","content_hash":"4f2b5a203d6a7b5e38176dd6ef68afb4b7d0e11889718381e28bf006f4e83a16","title":"Export deduplication breaks when JSONL and export_hashes table diverge","description":"## Problem\n\nThe export deduplication feature (timestamp-only skipping) breaks when the JSONL file and export_hashes table get out of sync, causing exports to skip issues that aren't actually in the file.\n\n## Symptoms\n\n- `bd export` reports \"Skipped 128 issue(s) with timestamp-only changes\"\n- JSONL file only has 38 lines but DB has 149 issues\n- export_hashes table has 149 entries\n- Auto-import doesn't trigger (hash matches despite missing data)\n- Two repos on same commit show different issue counts\n\n## Root Cause\n\nshouldSkipExport() in autoflush.go compares current issue hash with stored export_hashes entry. If they match, it skips export assuming the issue is already in the JSONL.\n\nThis assumption fails when:\n1. Git operations (pull, reset, checkout) change JSONL without clearing export_hashes\n2. Manual JSONL edits or corruption\n3. Import operations that modify DB but don't update export_hashes\n4. Partial exports that update export_hashes but don't complete\n\n## Impact\n\n- **Critical data loss risk**: Issues appear to be tracked but aren't persisted to git\n- Breaks multi-repo sync (root cause of today's debugging session)\n- Auto-import fails to detect staleness (hash matches despite missing data)\n- Silent data corruption (no error messages, just missing issues)\n\n## Reproduction\n\n1. Have DB with 149 issues, all in export_hashes table\n2. Truncate JSONL to 38 lines (simulate git reset or corruption)\n3. Run `bd export` - it skips 128 issues\n4. JSONL still has only 38 lines but export thinks it succeeded\n\n## Current Workaround\n\n```bash\nsqlite3 .beads/beads.db \"DELETE FROM export_hashes\"\nbd export -o .beads/beads.jsonl\n```\n\n## Proposed Solutions\n\n**Option 1: Verify JSONL integrity before skipping**\n- Count lines in JSONL, compare with export_hashes count\n- If mismatch, clear export_hashes and force full export\n- Safe but adds I/O overhead\n\n**Option 2: Hash-based JSONL validation**\n- Store hash of entire JSONL file in metadata\n- Before export, check if JSONL hash matches\n- If mismatch, clear export_hashes\n- More efficient, detects any JSONL corruption\n\n**Option 3: Disable timestamp-only deduplication**\n- Remove the feature entirely\n- Always export all issues\n- Simplest and safest, but creates larger git commits\n\n**Option 4: Clear export_hashes on git operations**\n- Add post-merge hook to clear export_hashes\n- Clear on any import operation\n- Defensive approach but may over-clear\n\n## Recommended Fix\n\nCombination of Options 2 + 4:\n1. Store JSONL file hash in metadata after export\n2. Check hash before export, clear export_hashes if mismatch \n3. Clear export_hashes on import operations\n4. Add `bd validate` check for JSONL/export_hashes sync\n\n## Files Involved\n\n- cmd/bd/autoflush.go (shouldSkipExport)\n- cmd/bd/export.go (export with deduplication)\n- internal/storage/sqlite/metadata.go (export_hashes table)","notes":"## Recovery Session (2025-10-29 21:30)\n\n### What Happened\n- Created 14 new hash ID issues (bd-f8b764c9 through bd-f8b764c9.1) \n- bd sync appeared to succeed\n- Canonical repo (~/src/beads): 162 issues in DB + JSONL ✓\n- Secondary repo (fred/beads): Only 145 issues vs 162 in canonical ✗\n- Both repos on same git commit but different issue counts!\n\n### Bug Manifestation During Recovery\n\n1. **Initial state**: fred/beads had 145 issues, 145 lines in JSONL, 145 export_hashes entries\n\n2. **After git reset --hard origin/main**: \n - JSONL: 162 lines (from git)\n - DB: 150 issues (auto-import partially worked)\n - Auto-import failed with UNIQUE constraint error\n\n3. **After manual import --resolve-collisions**:\n - DB: 160 issues\n - JSONL: Still 162 lines\n - export_hashes: 159 entries\n\n4. **After bd export**: \n - **JSONL reduced to 17 lines!** ← The bug in action\n - export_hashes: 159 entries (skipped exporting 142 issues)\n - Silent data loss - no error message\n\n5. **After clearing export_hashes and re-export**:\n - JSONL: 159 lines (missing 3 issues still)\n - DB: 159 issues\n - Still diverged from canonical\n\n### The Bug Loop\nOnce export_hashes and JSONL diverge:\n- Export skips issues already in export_hashes\n- But those issues aren't actually in JSONL\n- This creates corrupt JSONL with missing issues\n- Auto-import can't detect the problem (file hash matches what was exported)\n- Data is lost with no error messages\n\n### Recovery Solution\nCouldn't break the loop with export alone. Had to:\n1. Copy .beads/beads.db from canonical repo\n2. Clear export_hashes\n3. Full re-export\n4. Finally converged to 162 issues\n\n### Key Learnings\n\n1. **The bug is worse than we thought**: It can create corrupt exports (17 lines instead of 162!)\n\n2. **Auto-import can't save you**: Once export is corrupt, auto-import just imports the corrupt data\n\n3. **Silent failure**: No warnings, no errors, just missing issues\n\n4. **Git operations trigger it**: git reset, git pull, etc. change JSONL without clearing export_hashes\n\n5. **Import operations populate export_hashes**: Even manual imports update export_hashes, setting up future export failures\n\n### Immediate Action Required\n\n**DISABLE EXPORT DEDUPLICATION NOW**\n\nThis feature is fundamentally broken and causes data loss. Should be disabled until properly fixed.\n\nQuick fix options:\n- Set environment variable to disable feature\n- Comment out shouldSkipExport check\n- Always clear export_hashes before export\n- Add validation that DB count == JSONL line count before allowing export\n\n### Long-term Fix\n\nNeed Option 2 + 4 from proposed solutions:\n1. Store JSONL file hash after every successful export\n2. Before export, verify JSONL hash matches expected\n3. If mismatch, log WARNING and clear export_hashes\n4. Clear export_hashes on every import operation\n5. Add git post-merge hook to clear export_hashes\n6. Add `bd validate` command to detect divergence\n","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-10-29T23:05:13.959435-07:00","updated_at":"2025-10-30T17:12:58.207148-07:00","closed_at":"2025-10-29T21:57:03.06641-07:00"} {"id":"bd-5e1f","content_hash":"5b0aa7a2f651393bc13c46c172828acc4306d22d749ff71fbae96f0d25741847","title":"Issue with desc","description":"This is a description","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-10-31T21:41:11.128718-07:00","updated_at":"2025-11-02T15:58:53.554407-08:00","closed_at":"2025-11-02T15:58:53.55441-08:00"} {"id":"bd-5f26","content_hash":"75bc96be4d465a5eb39bdf0b636c42cdd7b8ac7daf90b47b7b2a015991b87512","title":"Refactor daemon.go into internal/daemonrunner","description":"Extract daemon runtime from daemon.go (1,565 lines) into internal/daemonrunner with focused modules: config.go, daemon.go, process.go, rpc_server.go, sync.go, git.go. Keep cobra command thin.","design":"New structure:\n- internal/daemonrunner/config.go: Config struct\n- internal/daemonrunner/daemon.go: Daemon struct + Start/Stop\n- internal/daemonrunner/process.go: PID/lock/socket handling\n- internal/daemonrunner/rpc_server.go: RPC lifecycle\n- internal/daemonrunner/sync.go: Export/import/commit/push logic\n- internal/daemonrunner/git.go: Git operations interface\n- cmd/bd/daemon.go: Thin cobra command","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-11-01T11:41:14.821017-07:00","updated_at":"2025-11-01T21:44:44.507747-07:00","closed_at":"2025-11-01T21:44:44.507747-07:00"} {"id":"bd-5f483051","content_hash":"d69f64f7f0bdc46a539dfe0b699a8977309c9c8d59f3e9beffbbe4484275a16b","title":"Implement bd resolve-conflicts (git merge conflicts in JSONL)","description":"Automatically detect and resolve git merge conflicts in .beads/issues.jsonl file.\n\nFeatures:\n- Detect conflict markers in JSONL\n- Parse conflicting issues from HEAD and BASE\n- Provide mechanical resolution (remap duplicate IDs)\n- Support AI-assisted resolution (requires internal/ai package)\n\nSee repair_commands.md lines 125-353 for design.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T19:37:55.722827-07:00","updated_at":"2025-10-30T17:12:58.179718-07:00"} -{"id":"bd-6049","content_hash":"16c54bc547f4ab180aee39efbb197709a47a39047f5bc2dd59e6e6b57ca8bc87","title":"bd doctor --json flag not working","description":"The --json flag on bd doctor command doesn't produce JSON output. It continues to show human-readable output instead. The flag is registered locally on doctorCmd but the code uses the global jsonOutput variable set by PersistentPreRun. Need to investigate why the flag isn't being honored.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-11-02T17:08:18.170428-08:00","updated_at":"2025-11-02T18:41:01.376783-08:00","closed_at":"2025-11-02T18:41:01.376786-08:00"} +{"id":"bd-6049","content_hash":"16c54bc547f4ab180aee39efbb197709a47a39047f5bc2dd59e6e6b57ca8bc87","title":"bd doctor --json flag not working","description":"The --json flag on bd doctor command doesn't produce JSON output. It continues to show human-readable output instead. The flag is registered locally on doctorCmd but the code uses the global jsonOutput variable set by PersistentPreRun. Need to investigate why the flag isn't being honored.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-11-02T17:08:18.170428-08:00","updated_at":"2025-11-02T18:41:01.376783-08:00","closed_at":"2025-11-02T18:41:01.376786-08:00","comments":[{"id":4,"issue_id":"bd-6049","author":"stevey","text":"Fixed by removing the local --json flag definition in doctor.go that was shadowing the persistent --json flag from main.go. The doctor command now correctly uses the global jsonOutput variable.","created_at":"2025-11-03T02:41:01Z"}]} {"id":"bd-6214875c","content_hash":"d4d20e71bbf5c08f1fe1ed07f67b7554167aa165d4972ea51b5cacc1b256c4c1","title":"Split internal/rpc/server.go into focused modules","description":"The file `internal/rpc/server.go` is 2,273 lines with 50+ methods, making it difficult to navigate and prone to merge conflicts. Split into 8 focused files with clear responsibilities.\n\nCurrent structure: Single 2,273-line file with:\n- Connection handling\n- Request routing\n- All 40+ RPC method implementations\n- Storage caching\n- Health checks \u0026 metrics\n- Cleanup loops\n\nTarget structure:\n```\ninternal/rpc/\n├── server.go # Core server, connection handling (~300 lines)\n├── methods_issue.go # Issue operations (~400 lines)\n├── methods_deps.go # Dependency operations (~200 lines)\n├── methods_labels.go # Label operations (~150 lines)\n├── methods_ready.go # Ready work queries (~150 lines)\n├── methods_compact.go # Compaction operations (~200 lines)\n├── methods_comments.go # Comment operations (~150 lines)\n├── storage_cache.go # Storage caching logic (~300 lines)\n└── health.go # Health \u0026 metrics (~200 lines)\n```\n\nMigration strategy:\n1. Create new files with appropriate methods\n2. Keep `server.go` as main file with core server logic\n3. Test incrementally after each file split\n4. Final verification with full test suite","acceptance_criteria":"- All 50 methods split into appropriate files\n- Each file \u003c500 LOC\n- All methods remain on `*Server` receiver (no behavior change)\n- All tests pass: `go test ./internal/rpc/...`\n- Verify daemon works: start daemon, run operations, check health\n- Update internal documentation if needed\n- No change to public API","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T14:21:37.51524-07:00","updated_at":"2025-10-30T17:12:58.2179-07:00","closed_at":"2025-10-28T14:11:04.399811-07:00"} {"id":"bd-6221bdcd","content_hash":"3bf15bc9e418180e1e91691261817c872330e182dbc1bcb756522faa42416667","title":"Improve cmd/bd test coverage (currently 20.2%)","description":"CLI commands need better test coverage. Focus on:\n- Command argument parsing\n- Error handling paths\n- Edge cases in create, update, close commands\n- Daemon commands\n- Import/export workflows","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-29T14:06:27.951656-07:00","updated_at":"2025-10-30T17:12:58.185819-07:00","dependencies":[{"issue_id":"bd-6221bdcd","depends_on_id":"bd-4d7fca8a","type":"blocks","created_at":"2025-10-29T19:52:05.532391-07:00","created_by":"import-remap"}]} {"id":"bd-627d","content_hash":"5b3d3d69ceac28dcbfbc2c7ea2f7a6ff2a3a02bc58ce02dcf6b05f8469e8bddc","title":"AI-supervised database migrations for safer schema evolution","description":"## Problem\n\nDatabase migrations can lose user data through edge cases that are hard to anticipate (e.g., GH #201 where bd migrate failed to set issue_prefix, or bd-d355a07d false positive data loss warnings). Since beads is designed to be run by AI agents, we should leverage AI to make migrations safer.\n\n## Current State\n\nMigrations run blindly with:\n- No pre-flight validation\n- No data integrity verification\n- No rollback on failure\n- Limited post-migration testing\n\nRecent issues:\n- GH #201: Migration didn't set issue_prefix config, breaking commands\n- bd-d355a07d: False positive \"data loss\" warnings on collision resolution\n- Users reported migration data loss (fixed but broader problem remains)\n\n## Proposal: AI-Supervised Migration Framework\n\nUse AI to supervise migrations through structured verification:\n\n### 1. Pre-Migration Analysis\n- AI reads migration code and current schema\n- Identifies potential data loss scenarios\n- Generates validation queries to verify assumptions\n- Creates snapshot queries for before/after comparison\n\n### 2. Migration Execution\n- Take database backup/snapshot\n- Run validation queries (pre-state)\n- Execute migration in transaction\n- Run validation queries (post-state)\n\n### 3. Post-Migration Verification\n- AI compares pre/post snapshots\n- Verifies data integrity invariants\n- Checks for unexpected data loss\n- Validates config completeness (like issue_prefix)\n\n### 4. Rollback on Anomalies\n- If AI detects data loss, rollback transaction\n- Present human-readable error report\n- Suggest fix before retrying\n\n## Example Flow\n\n```\n$ bd migrate\n\n→ Analyzing migration plan...\n→ AI identified 3 potential data loss scenarios\n→ Generating validation queries...\n→ Creating pre-migration snapshot...\n→ Running migration in transaction...\n→ Verifying post-migration state...\n✓ All 247 issues accounted for\n✓ Config table complete (issue_prefix: \"mcp\")\n✓ Dependencies intact (342 relationships verified)\n→ Migration successful!\n```\n\nIf something goes wrong:\n```\n$ bd migrate\n\n→ Analyzing migration plan...\n→ AI identified issue: Missing issue_prefix config after migration\n→ Recommendation: Add prefix detection step\n→ Aborting migration - database unchanged\n```\n\n## Implementation Ideas\n\n### A. Migration Validator Tool\nCreate `bd migrate --validate` that:\n- Simulates migration on copy of database\n- Uses AI to verify data integrity\n- Reports potential issues before real migration\n\n### B. Migration Test Generator\nAI generates test cases for migrations:\n- Edge cases (empty DB, large DB, missing config)\n- Data integrity checks\n- Regression tests\n\n### C. Migration Invariants\nDefine invariants that AI checks:\n- Issue count should not decrease (unless collision resolution)\n- All required config keys present\n- Foreign key relationships intact\n- No orphaned dependencies\n\n### D. Self-Healing Migrations\nAI detects incomplete migrations and suggests fixes:\n- Missing config values (like GH #201)\n- Orphaned data\n- Index inconsistencies\n\n## Benefits\n\n1. **Catch edge cases**: AI explores scenarios humans miss\n2. **Self-documenting**: AI explains what migration does\n3. **Agent-friendly**: Agents can run migrations confidently\n4. **Fewer rollbacks**: Detect issues before committing\n5. **Better testing**: AI generates comprehensive test suites\n\n## Open Questions\n\n1. Which AI model? (Fast: Haiku, Thorough: Sonnet/GPT-4)\n2. How to balance safety vs migration speed?\n3. Should AI validation be required or optional?\n4. How to handle offline scenarios (no API access)?\n5. What invariants should always be checked?\n\n## Related Work\n\n- bd-b245: Migration registry (makes migrations introspectable)\n- GH #201: issue_prefix migration bug (motivating example)\n- bd-d355a07d: False positive data loss warnings","design":"## Architecture: Agent-Supervised Migrations (Inversion of Control)\n\n**Key principle:** Beads provides observability and validation primitives. AI agents supervise using their own reasoning. Beads NEVER makes AI API calls.\n\n## Phase 1: Migration Invariants (Pure Validation)\n\nCreate `internal/storage/sqlite/migration_invariants.go`:\n\n```go\ntype MigrationInvariant struct {\n Name string\n Description string\n Check func(*sql.DB, *Snapshot) error\n}\n\ntype Snapshot struct {\n IssueCount int\n ConfigKeys []string\n DependencyCount int\n LabelCount int\n}\n\nvar invariants = []MigrationInvariant{\n {\n Name: \"required_config_present\",\n Description: \"Required config keys must exist\",\n Check: checkRequiredConfig, // Would have caught GH #201\n },\n {\n Name: \"foreign_keys_valid\",\n Description: \"No orphaned dependencies or labels\",\n Check: checkForeignKeys,\n },\n {\n Name: \"issue_count_stable\",\n Description: \"Issue count should not decrease unexpectedly\",\n Check: checkIssueCount,\n },\n}\n\nfunc checkRequiredConfig(db *sql.DB, snapshot *Snapshot) error {\n required := []string{\"issue_prefix\", \"schema_version\"}\n for _, key := range required {\n var value string\n err := db.QueryRow(\"SELECT value FROM config WHERE key = ?\", key).Scan(\u0026value)\n if err != nil || value == \"\" {\n return fmt.Errorf(\"required config key missing: %s\", key)\n }\n }\n return nil\n}\n```\n\n## Phase 2: Dry-Run \u0026 Inspection Tools\n\nAdd `bd migrate --dry-run --json`:\n\n```json\n{\n \"pending_migrations\": [\n {\"name\": \"dirty_issues_table\", \"description\": \"Adds dirty_issues table\"},\n {\"name\": \"content_hash_column\", \"description\": \"Adds content_hash for collision resolution\"}\n ],\n \"current_state\": {\n \"schema_version\": \"0.9.9\",\n \"issue_count\": 247,\n \"config\": {\"schema_version\": \"0.9.9\"},\n \"missing_config\": [\"issue_prefix\"]\n },\n \"warnings\": [\n \"issue_prefix config not set - may break commands after migration\"\n ],\n \"invariants_to_check\": [\n \"required_config_present\",\n \"foreign_keys_valid\",\n \"issue_count_stable\"\n ]\n}\n```\n\nAdd `bd info --schema --json`:\n\n```json\n{\n \"tables\": [\"issues\", \"dependencies\", \"labels\", \"config\"],\n \"schema_version\": \"0.9.9\",\n \"config\": {},\n \"sample_issue_ids\": [\"mcp-1\", \"mcp-2\"],\n \"detected_prefix\": \"mcp\"\n}\n```\n\n## Phase 3: Pre/Post Snapshots with Rollback\n\nUpdate `RunMigrations()`:\n\n```go\nfunc RunMigrations(db *sql.DB) error {\n // Capture pre-migration snapshot\n snapshot := captureSnapshot(db)\n \n // Run migrations in transaction\n tx, err := db.Begin()\n if err != nil {\n return err\n }\n defer tx.Rollback()\n \n for _, migration := range migrations {\n if err := migration.Func(tx); err != nil {\n return fmt.Errorf(\"migration %s failed: %w\", migration.Name, err)\n }\n }\n \n // Verify invariants before commit\n if err := verifyInvariants(tx, snapshot); err != nil {\n return fmt.Errorf(\"post-migration validation failed (rolled back): %w\", err)\n }\n \n return tx.Commit()\n}\n```\n\n## Phase 4: MCP Tools for Agent Supervision\n\nAdd to beads-mcp:\n\n```python\n@server.tool()\nasync def inspect_migration(workspace_root: str) -\u003e dict:\n \"\"\"Get migration plan and current state for agent analysis.\n \n Agent should:\n 1. Review pending migrations\n 2. Check for warnings (missing config, etc.)\n 3. Verify invariants will pass\n 4. Decide whether to run bd migrate\n \"\"\"\n result = run_bd([\"migrate\", \"--dry-run\", \"--json\"], workspace_root)\n return json.loads(result.stdout)\n\n@server.tool() \nasync def get_schema_info(workspace_root: str) -\u003e dict:\n \"\"\"Get current database schema for migration analysis.\"\"\"\n result = run_bd([\"info\", \"--schema\", \"--json\"], workspace_root)\n return json.loads(result.stdout)\n```\n\n## Agent Workflow Example\n\n```python\n# Agent detects user wants to migrate\nmigration_plan = inspect_migration(\"/path/to/workspace\")\n\n# Agent analyzes (using its own reasoning, no API calls from beads)\nif \"issue_prefix\" in migration_plan[\"missing_config\"]:\n schema = get_schema_info(\"/path/to/workspace\")\n detected_prefix = schema[\"detected_prefix\"]\n \n # Agent fixes issue before migration\n run_bd([\"config\", \"set\", \"issue_prefix\", detected_prefix])\n \n# Now safe to migrate\nrun_bd([\"migrate\"])\n```\n\n## What Beads Provides\n\n✅ Deterministic validation (invariants)\n✅ Structured inspection (--dry-run, --explain)\n✅ Rollback on invariant failure\n✅ JSON output for agent parsing\n\n## What Beads Does NOT Do\n\n❌ No AI API calls\n❌ No external model access\n❌ No agent invocation\n\nAgents supervise migrations using their own reasoning and the inspection tools beads provides.","acceptance_criteria":"Phase 1: Migration invariants implemented and tested, checked after every migration, clear error messages when invariants fail.\n\nPhase 2: Snapshot capture before migrations, comparison after, rollback on verification failure.\n\nPhase 3 (stretch): AI validation optional flag implemented, AI can analyze migration code and generate custom validation queries.\n\nPhase 4 (stretch): Migration test fixtures created, all fixtures pass migrations, CI runs migration tests.","notes":"## Progress\n\n### ✅ Phase 1: Migration Invariants (COMPLETED)\n\n**Implemented:**\n- Created internal/storage/sqlite/migration_invariants.go with 3 invariants\n- Updated RunMigrations() to verify invariants after migrations\n- All tests pass ✓\n\n### ✅ Phase 2: Inspection Tools (COMPLETED \u0026 PUSHED)\n\n**Commit:** 1abe4e7 - \"Add migration inspection tools for AI agents (bd-627d Phase 2)\"\n\n**Implemented:**\n1. ✅ bd migrate --inspect --json - Shows migration plan\n2. ✅ bd info --schema --json - Returns schema details\n3. ✅ Migration warnings system\n4. ✅ Documentation updated in AGENTS.md\n5. ✅ All tests pass\n\n### ✅ Phase 3: MCP Tools (COMPLETED \u0026 PUSHED)\n\n**Commit:** 2493693 - \"Add MCP tools for migration inspection (bd-627d Phase 3)\"\n\n**Implemented:**\n1. ✅ inspect_migration(workspace_root) tool in beads-mcp\n2. ✅ get_schema_info(workspace_root) tool in beads-mcp\n3. ✅ Abstract methods in BdClientBase\n4. ✅ CLI client implementations\n5. ✅ All tests pass\n\n**All phases complete!** Migration inspection fully integrated into MCP server.","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-11-02T12:57:10.722048-08:00","updated_at":"2025-11-02T14:31:25.095296-08:00","closed_at":"2025-11-02T14:31:25.095308-08:00"}