johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

bd sync: 2025-10-29 11:50:20

Browse Source
This commit is contained in:
Steve Yegge
2025-10-29 11:50:20 -07:00
parent 7d0cccdccb
commit 2f11eaa54b
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.beads/beads.jsonl
View File

@@ -91,7 +91,7 @@
{"id":"bd-8","title":"Daemon fails to auto-import after git pull updates JSONL","description":"After git pull updates .beads/issues.jsonl, daemon doesn't automatically re-import changes, causing stale data to be shown until next sync cycle (up to 5 minutes).\n\nReproduction:\n1. Repo A: Close issues, export, commit, push\n2. Repo B: git pull (successfully updates .beads/issues.jsonl)\n3. bd show \u003cissue\u003e shows OLD status from daemon's SQLite db\n4. JSONL on disk has correct new status\n\nRoot cause: Daemon sync cycle runs on timer (5min). When user manually runs git pull, daemon doesn't detect JSONL was updated externally and continues serving stale data from SQLite.\n\nImpact:\n- High for AI agents using beads in git workflows\n- Breaks fundamental git-as-source-of-truth model\n- Confusing UX: git log shows commit, bd shows old state\n- Data consistency issues between JSONL and daemon\n\nSee WYVERN_SYNC_ISSUE.md for full analysis.","design":"Three possible solutions:\n\nOption 1: Auto-detect and re-import (recommended)\n- Before serving any bd command, check if .beads/issues.jsonl mtime \u003e last import time\n- If newer, auto-import before processing request\n- Fast check, minimal overhead\n\nOption 2: File watcher in daemon\n- Daemon watches .beads/issues.jsonl for mtime changes\n- Auto-imports when file changes\n- More complex, requires file watching infrastructure\n\nOption 3: Explicit sync command\n- User runs `bd sync` after git pull\n- Manual, error-prone, defeats automation\n\nRecommended: Option 1 (auto-detect) + Option 3 (explicit sync) as fallback.","acceptance_criteria":"1. After git pull updates .beads/issues.jsonl, next bd command sees fresh data\n2. No manual import or daemon restart required\n3. Performance impact \u003c 10ms per command (mtime check is fast)\n4. Works in both daemon and non-daemon modes\n5. Test: Two repo clones, update in one, pull in other, verify immediate sync","notes":"**Current Status (2025-10-26):**\n\n✅ **Completed (bd-128):**\n- Created internal/autoimport package with staleness detection\n- Daemon can detect when JSONL is newer than last import\n- Infrastructure exists to call import logic\n\n❌ **Remaining Work:**\nThe daemon's importFunc in server.go (line 2096-2102) is a stub that just logs a notice. It needs to actually import the issues.\n\n**Problem:** \n- importIssuesCore is in cmd/bd package, not accessible from internal/rpc\n- daemon's handleImport() returns 'not yet implemented' error\n\n**Two approaches:**\n1. Move importIssuesCore to internal/import package (shares with daemon)\n2. Use storage layer directly in daemon (create/update issues via Storage interface)\n\n**Blocker:** \nThis is the critical bug causing data corruption:\n- Agent A pushes changes\n- Agent B does git pull\n- Agent B's daemon serves stale SQLite data\n- Agent B exports stale data back to JSONL, overwriting Agent A's changes\n- Agent B pushes, losing Agent A's work\n\n**Next Steps:**\n1. Choose approach (probably #1 - move importIssuesCore to internal/import)\n2. Implement real importFunc in daemon's checkAndAutoImportIfStale()\n3. Test with two-repo scenario (push from A, pull in B, verify B sees changes)\n4. Ensure no data corruption in multi-agent workflows","status":"in_progress","priority":0,"issue_type":"epic","created_at":"2025-10-25T23:13:12.270766-07:00","updated_at":"2025-10-27T22:22:23.815209-07:00"}
{"id":"bd-80","title":"Add mutation channel to internal/rpc/server.go","description":"Add mutationChan chan MutationEvent to Server struct. Emit events on CreateIssue, UpdateIssue, DeleteIssue, AddComment. Non-blocking send with default case for full channel.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T16:20:02.433388-07:00","updated_at":"2025-10-29T11:22:18.314571-07:00","closed_at":"2025-10-29T11:22:18.314571-07:00"}
{"id":"bd-81","title":"Add BEADS_DAEMON_MODE flag handling","description":"Add environment variable BEADS_DAEMON_MODE (values: poll, events). Default to 'poll' for Phase 1. Wire into daemon startup to select runEventLoop vs runEventDrivenLoop.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T16:20:02.433638-07:00","updated_at":"2025-10-28T16:20:02.433638-07:00","closed_at":"2025-10-28T12:31:47.819136-07:00"}
{"id":"bd-82","title":"Unit tests for Debouncer","description":"Test debouncer batches multiple triggers into single action. Test timer reset on subsequent triggers. Test cancel during wait. Test thread safety.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T16:20:02.433902-07:00","updated_at":"2025-10-28T16:20:02.433902-07:00","dependencies":[{"issue_id":"bd-82","depends_on_id":"bd-85","type":"parent-child","created_at":"2025-10-29T11:26:40.599533-07:00","created_by":"daemon"}]}
{"id":"bd-82","title":"Unit tests for Debouncer","description":"Test debouncer batches multiple triggers into single action. Test timer reset on subsequent triggers. Test cancel during wait. Test thread safety.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T16:20:02.433902-07:00","updated_at":"2025-10-29T11:50:20.357598-07:00","closed_at":"2025-10-29T11:50:20.357598-07:00","dependencies":[{"issue_id":"bd-82","depends_on_id":"bd-85","type":"parent-child","created_at":"2025-10-29T11:26:40.599533-07:00","created_by":"daemon"}]}
{"id":"bd-83","title":"Stress test: event storm handling","description":"Simulate 100+ rapid JSONL writes. Verify debouncer batches to single import. Verify no data loss. Test daemon stability.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T16:20:02.434221-07:00","updated_at":"2025-10-28T16:20:02.434221-07:00","dependencies":[{"issue_id":"bd-83","depends_on_id":"bd-85","type":"parent-child","created_at":"2025-10-29T11:26:40.600686-07:00","created_by":"daemon"}]}
{"id":"bd-84","title":"Remove unreachable utility functions","description":"Several small utility functions are unreachable:\n\nFiles to clean:\n1. `internal/storage/sqlite/hash.go` - `computeIssueContentHash` (line 17)\n - Check if entire file can be deleted if only contains this function\n\n2. `internal/config/config.go` - `FileUsed` (line 151)\n - Delete unused config helper\n\n3. `cmd/bd/git_sync_test.go` - `verifyIssueOpen` (line 300)\n - Delete dead test helper\n\n4. `internal/compact/haiku.go` - `HaikuClient.SummarizeTier2` (line 81)\n - Tier 2 summarization not implemented\n - Options: implement feature OR delete method\n\nImpact: Removes 50-100 LOC depending on decisions","acceptance_criteria":"- Remove unreachable functions\n- If entire files can be deleted (like hash.go), delete them\n- For SummarizeTier2: decide to implement or delete, document decision\n- All tests pass: `go test ./...`\n- Verify no callers exist for each function","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-28T16:20:02.434573-07:00","updated_at":"2025-10-28T16:20:02.434573-07:00"}
{"id":"bd-85","title":"Event-driven daemon architecture","description":"Replace 5-second polling sync loop with event-driven architecture that reacts instantly to changes. Eliminates stale data issues while reducing CPU ~60%. Key components: FileWatcher (fsnotify), Debouncer (500ms), RPC mutation events, optional git hooks. Target latency: \u003c500ms (vs 5000ms). See event_driven_daemon.md for full design.","notes":"## Implementation Progress\n\n**Completed:**\n1. ✅ Mutation events infrastructure (bd-80 equivalent)\n - MutationEvent channel in RPC server\n - Events emitted for all write operations: create, update, close, label add/remove, dep add/remove, comment add\n - Non-blocking emission with dropped event counter\n\n2. ✅ FileWatcher with fsnotify (bd-78 related)\n - Watches .beads/issues.jsonl and .git/refs/heads\n - 500ms debounce\n - Polling fallback if fsnotify unavailable\n\n3. ✅ Debouncer (bd-82 equivalent)\n - 500ms debounce for both export and import triggers\n - Thread-safe trigger/cancel\n\n4. ✅ Separate export-only and import-only functions\n - createExportFunc(): exports + optional commit/push (no pull/import)\n - createAutoImportFunc(): pull + import (no export)\n - Target latency \u003c500ms achieved by avoiding full sync\n\n5. ✅ Dropped events safety net (bd-83 related)\n - Atomic counter tracks dropped mutation events\n - 60-second health check triggers export if events were dropped\n - Prevents silent data loss from event storms\n\n**Still Needed:**\n- Platform-specific tests (bd-75)\n- Integration test for mutation→export latency (bd-77)\n- Unit tests for FileWatcher (bd-78)\n- Unit tests for Debouncer (bd-82)\n- Event storm stress test (bd-83)\n- Documentation update (bd-79)\n\n**Next Steps:**\nAdd comprehensive test coverage before enabling events mode by default.","status":"open","priority":1,"issue_type":"epic","created_at":"2025-10-28T16:30:27.39845-07:00","updated_at":"2025-10-29T11:30:59.851665-07:00"}