From 4fd1e20091161093ff2584de64961eb544d9019a Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Fri, 31 Oct 2025 20:28:38 -0700 Subject: [PATCH] bd sync: 2025-10-31 20:28:38 --- .beads/beads.jsonl | 1 + 1 file changed, 1 insertion(+) diff --git a/.beads/beads.jsonl b/.beads/beads.jsonl index a3a96177..d7fe15d1 100644 --- a/.beads/beads.jsonl +++ b/.beads/beads.jsonl @@ -6,6 +6,7 @@ {"id":"bd-09b5f2f5","content_hash":"f2eadd22bb585b0a14daff98029f8f43faec4163a369fb91b4329ec5800eae22","title":"Daemon fails to auto-import after git pull updates JSONL","description":"After git pull updates .beads/issues.jsonl, daemon doesn't automatically re-import changes, causing stale data to be shown until next sync cycle (up to 5 minutes).\n\nReproduction:\n1. Repo A: Close issues, export, commit, push\n2. Repo B: git pull (successfully updates .beads/issues.jsonl)\n3. bd show \u003cissue\u003e shows OLD status from daemon's SQLite db\n4. JSONL on disk has correct new status\n\nRoot cause: Daemon sync cycle runs on timer (5min). When user manually runs git pull, daemon doesn't detect JSONL was updated externally and continues serving stale data from SQLite.\n\nImpact:\n- High for AI agents using beads in git workflows\n- Breaks fundamental git-as-source-of-truth model\n- Confusing UX: git log shows commit, bd shows old state\n- Data consistency issues between JSONL and daemon\n\nSee WYVERN_SYNC_ISSUE.md for full analysis.","design":"Three possible solutions:\n\nOption 1: Auto-detect and re-import (recommended)\n- Before serving any bd command, check if .beads/issues.jsonl mtime \u003e last import time\n- If newer, auto-import before processing request\n- Fast check, minimal overhead\n\nOption 2: File watcher in daemon\n- Daemon watches .beads/issues.jsonl for mtime changes\n- Auto-imports when file changes\n- More complex, requires file watching infrastructure\n\nOption 3: Explicit sync command\n- User runs `bd sync` after git pull\n- Manual, error-prone, defeats automation\n\nRecommended: Option 1 (auto-detect) + Option 3 (explicit sync) as fallback.","acceptance_criteria":"1. After git pull updates .beads/issues.jsonl, next bd command sees fresh data\n2. No manual import or daemon restart required\n3. Performance impact \u003c 10ms per command (mtime check is fast)\n4. Works in both daemon and non-daemon modes\n5. Test: Two repo clones, update in one, pull in other, verify immediate sync","notes":"**Current Status (2025-10-26):**\n\n✅ **Completed (bd-a5251b1a):**\n- Created internal/autoimport package with staleness detection\n- Daemon can detect when JSONL is newer than last import\n- Infrastructure exists to call import logic\n\n❌ **Remaining Work:**\nThe daemon's importFunc in server.go (line 2096-2102) is a stub that just logs a notice. It needs to actually import the issues.\n\n**Problem:** \n- importIssuesCore is in cmd/bd package, not accessible from internal/rpc\n- daemon's handleImport() returns 'not yet implemented' error\n\n**Two approaches:**\n1. Move importIssuesCore to internal/import package (shares with daemon)\n2. Use storage layer directly in daemon (create/update issues via Storage interface)\n\n**Blocker:** \nThis is the critical bug causing data corruption:\n- Agent A pushes changes\n- Agent B does git pull\n- Agent B's daemon serves stale SQLite data\n- Agent B exports stale data back to JSONL, overwriting Agent A's changes\n- Agent B pushes, losing Agent A's work\n\n**Next Steps:**\n1. Choose approach (probably #1 - move importIssuesCore to internal/import)\n2. Implement real importFunc in daemon's checkAndAutoImportIfStale()\n3. Test with two-repo scenario (push from A, pull in B, verify B sees changes)\n4. Ensure no data corruption in multi-agent workflows","status":"in_progress","priority":0,"issue_type":"epic","created_at":"2025-10-25T23:13:12.270766-07:00","updated_at":"2025-10-30T17:12:58.223333-07:00"} {"id":"bd-0dcea000","title":"Add tests for internal/importer package","description":"Currently 0.0% coverage. Need tests for JSONL import logic including collision detection and resolution.","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-29T14:06:21.071024-07:00","updated_at":"2025-10-30T17:12:58.183211-07:00","dependencies":[{"issue_id":"bd-0dcea000","depends_on_id":"bd-cbed9619.5","type":"blocks","created_at":"2025-10-29T19:52:05.531279-07:00","created_by":"import-remap"},{"issue_id":"bd-0dcea000","depends_on_id":"bd-cbed9619.4","type":"blocks","created_at":"2025-10-29T19:52:05.53166-07:00","created_by":"import-remap"}]} {"id":"bd-0e1f2b1b","content_hash":"c0b1677fe3f4aa3f395ae4d79bff5362632d5db26477bf571c09f9177b8741ef","title":"Event-driven daemon architecture","description":"Replace 5-second polling sync loop with event-driven architecture that reacts instantly to changes. Eliminates stale data issues while reducing CPU ~60%. Key components: FileWatcher (fsnotify), Debouncer (500ms), RPC mutation events, optional git hooks. Target latency: \u003c500ms (vs 5000ms). See event_driven_daemon.md for full design.","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-10-28T16:20:02.430479-07:00","updated_at":"2025-10-30T17:12:58.221424-07:00","closed_at":"2025-10-28T16:30:26.631191-07:00"} +{"id":"bd-12c2","content_hash":"caa7a5f6e61a9b1c872f5462e53ed0ecdedc8835265ba6b7c7ef2100fa6448ae","title":"Add comprehensive tests for show.go commands (show, update, edit, close)","description":"Need to add tests for cmd/bd/show.go which contains show, update, edit, and close commands.\n\n**Challenge**: The existing test patterns use rootCmd.SetArgs() and rootCmd.Execute(), but the global `store` variable needs to match what the commands use. Initial attempt created tests that failed with \"no issue found\" because the test's store instance wasn't the same as the command's store.\n\n**Files to test**:\n- show.go (contains showCmd, updateCmd, editCmd, closeCmd)\n\n**Coverage needed**:\n- show command (single issue, multiple issues, JSON output, with dependencies, with labels, with compaction)\n- update command (status, priority, title, assignee, description, multiple fields, multiple issues)\n- edit command (requires $EDITOR, may need mocking)\n- close command (single issue, multiple issues, with reason, JSON output)\n\n**Test approach**:\n1. Study working test patterns in init_test.go, list_test.go, etc.\n2. Ensure BEADS_NO_DAEMON=1 is set\n3. Properly initialize database with bd init\n4. Use the command's global store, not a separate instance\n5. May need to reset global state between tests\n\n**Success criteria**: \n- All test functions pass\n- Coverage for show.go increases significantly\n- Tests follow existing patterns in cmd/bd/*_test.go","status":"closed","priority":0,"issue_type":"task","created_at":"2025-10-31T20:08:40.545173-07:00","updated_at":"2025-10-31T20:19:22.411066-07:00","closed_at":"2025-10-31T20:19:22.411066-07:00"} {"id":"bd-17fa2d21","content_hash":"139511c299c2fa977d11d5496245b23f4c27fe785fd4fc94470a35b0e4a01a0a","title":"Batch test 2","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-29T15:29:01.877052-07:00","updated_at":"2025-10-31T12:00:43.183657-07:00","closed_at":"2025-10-31T12:00:43.183657-07:00"} {"id":"bd-1863608e","content_hash":"697d09d1b24bc5120e6c8f2f94828a3c490f1432ee3ca243aa492fe465e04039","title":"Add TestNWayCollision for 5+ clones","description":"## Overview\nAdd comprehensive tests for N-way (5+) collision resolution to verify the solution scales beyond 3 clones.\n\n## Purpose\nWhile TestThreeCloneCollision validates the basic N-way case, we need to verify:\n1. Solution scales to arbitrary N\n2. Performance is acceptable with more clones\n3. Convergence time is bounded\n4. No edge cases in larger collision groups\n\n## Implementation Tasks\n\n### 1. Create TestFiveCloneCollision\nFile: beads_twoclone_test.go (or new beads_nway_test.go)\n\n```go\nfunc TestFiveCloneCollision(t *testing.T) {\n // Test with 5 clones creating same ID with different content\n // Verify all 5 clones converge after sync rounds\n \n t.Run(\"SequentialSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"A\", \"B\", \"C\", \"D\", \"E\")\n })\n \n t.Run(\"ReverseSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"E\", \"D\", \"C\", \"B\", \"A\")\n })\n \n t.Run(\"RandomSync\", func(t *testing.T) {\n testNCloneCollision(t, 5, \"C\", \"A\", \"E\", \"B\", \"D\")\n })\n}\n```\n\n### 2. Implement generalized testNCloneCollision\nGeneralize the 3-clone test to handle arbitrary N:\n\n```go\nfunc testNCloneCollision(t *testing.T, numClones int, syncOrder ...string) {\n t.Helper()\n \n if len(syncOrder) != numClones {\n t.Fatalf(\"syncOrder length (%d) must match numClones (%d)\", \n len(syncOrder), numClones)\n }\n \n tmpDir := t.TempDir()\n \n // Setup remote and N clones\n remoteDir := setupBareRepo(t, tmpDir)\n cloneDirs := make(map[string]string)\n \n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n cloneDirs[name] = setupClone(t, tmpDir, remoteDir, name)\n }\n \n // Each clone creates issue with same ID but different content\n for name, dir := range cloneDirs {\n createIssue(t, dir, fmt.Sprintf(\"Issue from clone %s\", name))\n }\n \n // Sync in specified order\n for _, name := range syncOrder {\n syncClone(t, cloneDirs[name], name)\n }\n \n // Final pull for convergence\n for name, dir := range cloneDirs {\n finalPull(t, dir, name)\n }\n \n // Verify all clones have all N issues\n expectedTitles := make(map[string]bool)\n for i := 0; i \u003c numClones; i++ {\n name := string(rune('A' + i))\n expectedTitles[fmt.Sprintf(\"Issue from clone %s\", name)] = true\n }\n \n for name, dir := range cloneDirs {\n titles := getTitles(t, dir)\n if !compareTitleSets(titles, expectedTitles) {\n t.Errorf(\"Clone %s missing issues: expected %v, got %v\", \n name, expectedTitles, titles)\n }\n }\n \n t.Log(\"✓ All\", numClones, \"clones converged successfully\")\n}\n```\n\n### 3. Add performance benchmarks\nTest convergence time and memory usage:\n\n```go\nfunc BenchmarkNWayCollision(b *testing.B) {\n for _, n := range []int{3, 5, 10, 20} {\n b.Run(fmt.Sprintf(\"N=%d\", n), func(b *testing.B) {\n for i := 0; i \u003c b.N; i++ {\n // Run N-way collision and measure time\n testNCloneCollisionBench(b, n)\n }\n })\n }\n}\n```\n\n### 4. Add convergence time tests\nVerify bounded convergence:\n\n```go\nfunc TestConvergenceTime(t *testing.T) {\n // Test that convergence happens within expected rounds\n // For N clones, should converge in at most N-1 sync rounds\n \n for n := 3; n \u003c= 10; n++ {\n t.Run(fmt.Sprintf(\"N=%d\", n), func(t *testing.T) {\n rounds := measureConvergenceRounds(t, n)\n maxExpected := n - 1\n if rounds \u003e maxExpected {\n t.Errorf(\"Convergence took %d rounds, expected ≤ %d\", \n rounds, maxExpected)\n }\n })\n }\n}\n```\n\n### 5. Add edge case tests\nTest boundary conditions:\n- All N clones have identical content (dedup works)\n- N-1 clones have same content, 1 differs\n- All N clones have unique content\n- Mix of collisions and non-collisions\n\n## Acceptance Criteria\n- TestFiveCloneCollision passes with all sync orders\n- All 5 clones converge to identical content\n- Performance is acceptable (\u003c 5 seconds for 5 clones)\n- Convergence time is bounded (≤ N-1 rounds)\n- Edge cases handled correctly\n- Benchmarks show scalability to 10+ clones\n\n## Files to Create/Modify\n- beads_twoclone_test.go or beads_nway_test.go\n- Add helper functions for N-clone setup\n\n## Testing Strategy\n\n### Test Matrix\n| N Clones | Sync Orders | Expected Result |\n|----------|-------------|-----------------|\n| 3 | A→B→C | Pass |\n| 3 | C→B→A | Pass |\n| 5 | A→B→C→D→E | Pass |\n| 5 | E→D→C→B→A | Pass |\n| 5 | Random | Pass |\n| 10 | Sequential | Pass |\n\n### Performance Targets\n- 3 clones: \u003c 2 seconds\n- 5 clones: \u003c 5 seconds\n- 10 clones: \u003c 15 seconds\n\n## Dependencies\n- Requires bd-cbed9619.5, bd-cbed9619.4, bd-0dcea000, bd-4d7fca8a to be completed\n- TestThreeCloneCollision must pass first\n\n## Success Metrics\n- All tests pass for N ∈ {3, 5, 10}\n- Convergence time scales linearly (O(N))\n- Memory usage reasonable (\u003c 100MB for 10 clones)\n- No data corruption or loss in any scenario","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-29T20:02:47.954306-07:00","updated_at":"2025-10-30T17:12:58.182217-07:00","closed_at":"2025-10-28T20:47:28.317007-07:00"} {"id":"bd-1c63eb84","content_hash":"178adb74f06c9a049ec5db6c406253005ee3460e7b732801e60fcee044986004","title":"Investigate jujutsu integration for beads","description":"Research and document how beads could integrate with jujutsu (jj), the next-generation VCS. Key areas to explore:\n- How jj's operation model differs from git (immutable operations, working-copy-as-commit)\n- JSONL sync strategy with jj's conflict resolution model\n- Daemon compatibility with jj's more frequent rewrites\n- Whether auto-import/export needs changes for jj workflows\n- Example configurations and documentation updates needed","status":"open","priority":3,"issue_type":"task","created_at":"2025-10-23T09:23:23.582009-07:00","updated_at":"2025-10-30T17:12:58.177733-07:00"}