johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

bd sync: 2025-10-28 12:31:47

Browse Source
This commit is contained in:
Steve Yegge
2025-10-28 12:31:47 -07:00
parent 96fb7ed29c
commit 97c676077d
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.beads/beads.jsonl
View File

@@ -43,13 +43,13 @@
{"id":"bd-48","title":"Replace getStorageForRequest with Direct Access","description":"Replace all getStorageForRequest(req) calls with s.storage","acceptance_criteria":"- No references to getStorageForRequest() in codebase (except in deleted file)\n- All handlers use s.storage directly\n- Code compiles without errors\n\nFiles to update:\n- internal/rpc/server_issues_epics.go (~8 calls)\n- internal/rpc/server_labels_deps_comments.go (~4 calls)\n- internal/rpc/server_compact.go (~2 calls)\n- internal/rpc/server_export_import_auto.go (~2 calls)\n- internal/rpc/server_routing_validation_diagnostics.go (~1 call)\n\nPattern: store, err := s.getStorageForRequest(req) → store := s.storage","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-27T23:20:10.393759-07:00","updated_at":"2025-10-27T23:20:10.393759-07:00"}
{"id":"bd-49","title":"Event-driven daemon architecture","description":"Replace 5-second polling sync loop with event-driven architecture that reacts instantly to changes. Eliminates stale data issues while reducing CPU ~60%. Key components: FileWatcher (fsnotify), Debouncer (500ms), RPC mutation events, optional git hooks. Target latency: \u003c500ms (vs 5000ms). See event_driven_daemon.md for full design.","status":"open","priority":1,"issue_type":"epic","created_at":"2025-10-28T12:01:46.480062-07:00","updated_at":"2025-10-28T12:01:46.480062-07:00"}
{"id":"bd-5","title":"Address gosec security warnings (102 issues)","description":"Security linter warnings: file permissions (0755 should be 0750), G304 file inclusion via variable, G204 subprocess launches. Many are false positives but should be reviewed.","design":"Review each gosec warning. Add exclusions for legitimate cases to .golangci.yml. Fix real security issues (overly permissive file modes).","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-25T13:47:10.719134-07:00","updated_at":"2025-10-27T22:22:23.814301-07:00"}
{"id":"bd-50","title":"Add BEADS_DAEMON_MODE flag handling","description":"Add environment variable BEADS_DAEMON_MODE (values: poll, events). Default to 'poll' for Phase 1. Wire into daemon startup to select runEventLoop vs runEventDrivenLoop.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.230043-07:00","updated_at":"2025-10-28T12:02:03.230043-07:00","dependencies":[{"issue_id":"bd-50","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.230666-07:00","created_by":"daemon"}]}
{"id":"bd-50","title":"Add BEADS_DAEMON_MODE flag handling","description":"Add environment variable BEADS_DAEMON_MODE (values: poll, events). Default to 'poll' for Phase 1. Wire into daemon startup to select runEventLoop vs runEventDrivenLoop.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.230043-07:00","updated_at":"2025-10-28T12:31:47.819136-07:00","closed_at":"2025-10-28T12:31:47.819136-07:00","dependencies":[{"issue_id":"bd-50","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.230666-07:00","created_by":"daemon"}]}
{"id":"bd-51","title":"Create cmd/bd/daemon_debouncer.go (~60 LOC)","description":"Implement Debouncer to batch rapid events into single action. Default 500ms, configurable via BEADS_DEBOUNCE_MS. Thread-safe with mutex.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.234539-07:00","updated_at":"2025-10-28T12:03:35.614191-07:00","closed_at":"2025-10-28T12:03:35.614191-07:00","dependencies":[{"issue_id":"bd-51","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.235218-07:00","created_by":"daemon"}]}
{"id":"bd-52","title":"Add fallback to polling on watcher failure","description":"Detect fsnotify.NewWatcher() errors and log warning. Auto-switch to polling mode with 5s ticker. Add BEADS_WATCHER_FALLBACK env var to control behavior.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.235717-07:00","updated_at":"2025-10-28T12:02:03.235717-07:00"}
{"id":"bd-53","title":"Create cmd/bd/daemon_watcher.go (~150 LOC)","description":"Implement FileWatcher using fsnotify to watch JSONL file and git refs. Handle platform differences (inotify/FSEvents/ReadDirectoryChangesW). Include edge case handling for file rename, event storm, watcher failure.","status":"in_progress","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.236475-07:00","updated_at":"2025-10-28T12:03:52.950843-07:00","dependencies":[{"issue_id":"bd-53","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.238539-07:00","created_by":"daemon"}]}
{"id":"bd-54","title":"Add fsnotify dependency to go.mod","description":"","status":"in_progress","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.239116-07:00","updated_at":"2025-10-28T12:03:26.102869-07:00","dependencies":[{"issue_id":"bd-54","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.239656-07:00","created_by":"daemon"}]}
{"id":"bd-55","title":"Create cmd/bd/daemon_event_loop.go (~200 LOC)","description":"Implement runEventDrivenLoop to replace polling ticker. Coordinate FileWatcher, mutation events, debouncer. Include health check ticker (60s) for daemon validation.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.23654-07:00","updated_at":"2025-10-28T12:02:03.23654-07:00","dependencies":[{"issue_id":"bd-55","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.242148-07:00","created_by":"daemon"}]}
{"id":"bd-56","title":"Add mutation channel to internal/rpc/server.go","description":"Add mutationChan chan MutationEvent to Server struct. Emit events on CreateIssue, UpdateIssue, DeleteIssue, AddComment. Non-blocking send with default case for full channel.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.23581-07:00","updated_at":"2025-10-28T12:02:03.23581-07:00","dependencies":[{"issue_id":"bd-56","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.246888-07:00","created_by":"daemon"}]}
{"id":"bd-55","title":"Create cmd/bd/daemon_event_loop.go (~200 LOC)","description":"Implement runEventDrivenLoop to replace polling ticker. Coordinate FileWatcher, mutation events, debouncer. Include health check ticker (60s) for daemon validation.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.23654-07:00","updated_at":"2025-10-28T12:30:44.067036-07:00","closed_at":"2025-10-28T12:30:44.067036-07:00","dependencies":[{"issue_id":"bd-55","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.242148-07:00","created_by":"daemon"}]}
{"id":"bd-56","title":"Add mutation channel to internal/rpc/server.go","description":"Add mutationChan chan MutationEvent to Server struct. Emit events on CreateIssue, UpdateIssue, DeleteIssue, AddComment. Non-blocking send with default case for full channel.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:03.23581-07:00","updated_at":"2025-10-28T12:29:53.32946-07:00","closed_at":"2025-10-28T12:29:53.32946-07:00","dependencies":[{"issue_id":"bd-56","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:03.246888-07:00","created_by":"daemon"}]}
{"id":"bd-57","title":"Platform tests: Linux, macOS, Windows","description":"Test event-driven mode on all platforms. Verify inotify (Linux), FSEvents (macOS), ReadDirectoryChangesW (Windows). Test fallback behavior on each.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:21.606053-07:00","updated_at":"2025-10-28T12:02:21.606053-07:00","dependencies":[{"issue_id":"bd-57","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:21.607758-07:00","created_by":"daemon"}]}
{"id":"bd-58","title":"Unit tests for Debouncer","description":"Test debouncer batches multiple triggers into single action. Test timer reset on subsequent triggers. Test cancel during wait. Test thread safety.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:21.6068-07:00","updated_at":"2025-10-28T12:02:21.6068-07:00","dependencies":[{"issue_id":"bd-58","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:21.60732-07:00","created_by":"daemon"}]}
{"id":"bd-59","title":"Stress test: event storm handling","description":"Simulate 100+ rapid JSONL writes. Verify debouncer batches to single import. Verify no data loss. Test daemon stability.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:21.616212-07:00","updated_at":"2025-10-28T12:02:21.616212-07:00","dependencies":[{"issue_id":"bd-59","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:21.616846-07:00","created_by":"daemon"}]}
@@ -59,6 +59,7 @@
{"id":"bd-62","title":"Integration test: mutation to export latency","description":"Measure time from bd create to JSONL update. Verify \u003c500ms latency. Test with multiple rapid mutations to verify batching.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:21.620034-07:00","updated_at":"2025-10-28T12:02:21.620034-07:00"}
{"id":"bd-63","title":"Unit tests for FileWatcher","description":"Test watcher detects JSONL changes. Test git ref changes trigger import. Test debounce integration. Test watcher recovery from file removal/rename.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:02:21.621407-07:00","updated_at":"2025-10-28T12:02:21.621407-07:00","dependencies":[{"issue_id":"bd-63","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:21.622215-07:00","created_by":"daemon"}]}
{"id":"bd-64","title":"Update AGENTS.md with event-driven mode","description":"Document BEADS_DAEMON_MODE env var. Explain opt-in during Phase 1. Add troubleshooting for watcher failures.","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-28T12:02:21.621626-07:00","updated_at":"2025-10-28T12:02:21.621626-07:00","dependencies":[{"issue_id":"bd-64","depends_on_id":"bd-49","type":"parent-child","created_at":"2025-10-28T12:02:21.623222-07:00","created_by":"daemon"}]}
{"id":"bd-65","title":"Add mutation channel to internal/rpc/server.go","description":"Add mutationChan chan MutationEvent to Server struct. Emit events on CreateIssue, UpdateIssue, DeleteIssue, AddComment. Non-blocking send with default case for full channel.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-28T12:28:28.685845-07:00","updated_at":"2025-10-28T12:28:28.685845-07:00"}
{"id":"bd-7","title":"Enforce daemon singleton per workspace with file locking","description":"Agent in ~/src/wyvern discovered 4 simultaneous daemon processes running, causing infinite directory recursion (.beads/.beads/.beads/...). Each daemon used relative paths and created nested .beads/ directories.\n\nRoot cause: No singleton enforcement. Multiple `bd daemon` processes can start in same workspace.\n\nExpected: One daemon per workspace (each workspace = separate .beads/ dir with bd.sock)\nActual: Multiple daemons can run simultaneously in same workspace\n\nNote: Separate git clones = separate workspaces = separate daemons (correct). Git worktrees share .beads/ and have known limitations (documented, use --no-daemon).","design":"Use flock (file locking) on daemon socket or database file to enforce singleton:\n\n1. On daemon start, attempt exclusive lock on .beads/bd.sock or .beads/daemon.lock\n2. If lock held by another process, refuse to start (exit with clear error)\n3. Hold lock for lifetime of daemon process\n4. Release lock on daemon shutdown\n\nAlternative: Use PID file with stale detection (check if PID is still running)\n\nImplementation location: Daemon startup code in cmd/bd/ or internal/daemon/","acceptance_criteria":"1. Starting second daemon process in same workspace fails with clear error\n2. Test: Start daemon, attempt second start, verify failure\n3. Killing daemon releases lock, allowing new daemon to start\n4. No infinite .beads/ directory recursion possible\n5. Works correctly with auto-start mechanism","status":"in_progress","priority":0,"issue_type":"bug","created_at":"2025-10-25T23:13:12.269549-07:00","updated_at":"2025-10-27T22:22:23.814937-07:00"}
{"id":"bd-8","title":"Daemon fails to auto-import after git pull updates JSONL","description":"After git pull updates .beads/issues.jsonl, daemon doesn't automatically re-import changes, causing stale data to be shown until next sync cycle (up to 5 minutes).\n\nReproduction:\n1. Repo A: Close issues, export, commit, push\n2. Repo B: git pull (successfully updates .beads/issues.jsonl)\n3. bd show \u003cissue\u003e shows OLD status from daemon's SQLite db\n4. JSONL on disk has correct new status\n\nRoot cause: Daemon sync cycle runs on timer (5min). When user manually runs git pull, daemon doesn't detect JSONL was updated externally and continues serving stale data from SQLite.\n\nImpact:\n- High for AI agents using beads in git workflows\n- Breaks fundamental git-as-source-of-truth model\n- Confusing UX: git log shows commit, bd shows old state\n- Data consistency issues between JSONL and daemon\n\nSee WYVERN_SYNC_ISSUE.md for full analysis.","design":"Three possible solutions:\n\nOption 1: Auto-detect and re-import (recommended)\n- Before serving any bd command, check if .beads/issues.jsonl mtime \u003e last import time\n- If newer, auto-import before processing request\n- Fast check, minimal overhead\n\nOption 2: File watcher in daemon\n- Daemon watches .beads/issues.jsonl for mtime changes\n- Auto-imports when file changes\n- More complex, requires file watching infrastructure\n\nOption 3: Explicit sync command\n- User runs `bd sync` after git pull\n- Manual, error-prone, defeats automation\n\nRecommended: Option 1 (auto-detect) + Option 3 (explicit sync) as fallback.","acceptance_criteria":"1. After git pull updates .beads/issues.jsonl, next bd command sees fresh data\n2. No manual import or daemon restart required\n3. Performance impact \u003c 10ms per command (mtime check is fast)\n4. Works in both daemon and non-daemon modes\n5. Test: Two repo clones, update in one, pull in other, verify immediate sync","notes":"**Current Status (2025-10-26):**\n\n✅ **Completed (bd-128):**\n- Created internal/autoimport package with staleness detection\n- Daemon can detect when JSONL is newer than last import\n- Infrastructure exists to call import logic\n\n❌ **Remaining Work:**\nThe daemon's importFunc in server.go (line 2096-2102) is a stub that just logs a notice. It needs to actually import the issues.\n\n**Problem:** \n- importIssuesCore is in cmd/bd package, not accessible from internal/rpc\n- daemon's handleImport() returns 'not yet implemented' error\n\n**Two approaches:**\n1. Move importIssuesCore to internal/import package (shares with daemon)\n2. Use storage layer directly in daemon (create/update issues via Storage interface)\n\n**Blocker:** \nThis is the critical bug causing data corruption:\n- Agent A pushes changes\n- Agent B does git pull\n- Agent B's daemon serves stale SQLite data\n- Agent B exports stale data back to JSONL, overwriting Agent A's changes\n- Agent B pushes, losing Agent A's work\n\n**Next Steps:**\n1. Choose approach (probably #1 - move importIssuesCore to internal/import)\n2. Implement real importFunc in daemon's checkAndAutoImportIfStale()\n3. Test with two-repo scenario (push from A, pull in B, verify B sees changes)\n4. Ensure no data corruption in multi-agent workflows","status":"in_progress","priority":0,"issue_type":"epic","created_at":"2025-10-25T23:13:12.270766-07:00","updated_at":"2025-10-27T22:22:23.815209-07:00"}
{"id":"bd-9","title":"Document bd edit command and verify MCP exclusion","description":"Follow-up from PR #152:\n1. Add \"bd edit\" to AGENTS.md with \"Humans only\" note\n2. Verify MCP server doesn't expose bd edit command\n3. Consider adding test for command registration","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-26T13:23:47.982295-07:00","updated_at":"2025-10-27T22:22:23.815469-07:00"}