johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add critical daemon and sync issues: singleton enforcement, exclusive locks, git pull auto-sync

Browse Source 
Amp-Thread-ID: https://ampcode.com/threads/T-1d43dd4a-cee3-40c5-b6aa-c071c664d093
Co-authored-by: Amp <amp@ampcode.com>
This commit is contained in:
Steve Yegge
2025-10-25 22:48:24 -07:00
parent ef5128fe24
commit 1e770a2f2e
2 changed files with 7 additions and 484 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.beads/beads.jsonl
View File

@@ -53,7 +53,14 @@
{"id":"bd-146","title":"Add configurable SortPolicy to GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.","design":"See SORT_POLICY_DESIGN.md for complete design.\n\nImplementation summary:\n1. Add SortPolicy type and constants (hybrid, priority, oldest)\n2. Add SortPolicy field to WorkFilter \n3. Implement buildOrderByClause() to generate SQL based on policy\n4. Default to hybrid for backwards compatibility\n5. Add --sort flag to bd ready command\n\nThis enables autonomous execution systems (like VC) to use strict priority ordering while preserving the current hybrid behavior for interactive use.","acceptance_criteria":"Unit tests verify each policy generates correct ORDER BY. Integration tests verify priority, hybrid, and oldest policies select issues in expected order. CLI bd ready --sort priority works. Empty SortPolicy defaults to hybrid (backwards compatible).","status":"closed","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:44:27.907777-07:00","updated_at":"2025-10-25T18:50:52.816914-07:00","closed_at":"2025-10-25T18:50:52.816914-07:00"}
{"id":"bd-147","title":"Implement configurable sort policy for GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different sorting strategies:\n- hybrid (default): Recent issues by priority, old by age\n- priority: Strict priority ordering for autonomous execution\n- oldest: Backlog clearing mode\n\nSolves issue where VC executor selects low-priority work instead of critical P0 work.","design":"See SORT_POLICY_DESIGN.md for complete design spec including:\n- Type definitions and constants\n- SQL ORDER BY clause generation\n- Testing strategy with test cases\n- CLI integration with --sort flag\n- Migration plan and backwards compatibility","acceptance_criteria":"- SortPolicy type added to types.go\n- buildOrderByClause() implemented in ready.go\n- Unit tests for all 3 policies pass\n- Integration tests verify priority selection order\n- bd ready --sort priority|oldest|hybrid works\n- Empty SortPolicy defaults to hybrid (backwards compatible)\n- Documentation updated in README.md and WORKFLOW.md","status":"closed","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:46:44.35198-07:00","updated_at":"2025-10-25T18:53:02.258745-07:00","closed_at":"2025-10-25T18:53:02.258745-07:00"}
{"id":"bd-148","title":"Add configurable SortPolicy to GetReadyWork","description":"Add SortPolicy field to WorkFilter to support different ordering strategies: hybrid (default), priority-first, oldest-first. See SORT_POLICY_DESIGN.md for full specification.","design":"See SORT_POLICY_DESIGN.md for complete design.\n\nImplementation summary:\n1. Add SortPolicy type and constants (hybrid, priority, oldest)\n2. Add SortPolicy field to WorkFilter \n3. Implement buildOrderByClause() to generate SQL based on policy\n4. Default to hybrid for backwards compatibility\n5. Add --sort flag to bd ready command\n\nThis enables autonomous execution systems (like VC) to use strict priority ordering while preserving the current hybrid behavior for interactive use.","acceptance_criteria":"Unit tests verify each policy generates correct ORDER BY. Integration tests verify priority, hybrid, and oldest policies select issues in expected order. CLI bd ready --sort priority works. Empty SortPolicy defaults to hybrid (backwards compatible).","status":"open","priority":1,"issue_type":"feature","created_at":"2025-10-25T18:51:11.560979-07:00","updated_at":"2025-10-25T18:51:11.560979-07:00"}
{"id":"bd-149","title":"Enforce daemon singleton per workspace with file locking","description":"Agent in ~/src/wyvern discovered 4 simultaneous daemon processes running, causing infinite directory recursion (.beads/.beads/.beads/...). Each daemon used relative paths and created nested .beads/ directories.\n\nRoot cause: No singleton enforcement. Multiple `bd daemon` processes can start in same workspace.\n\nExpected: One daemon per workspace (each workspace = separate .beads/ dir with bd.sock)\nActual: Multiple daemons can run simultaneously in same workspace\n\nNote: Separate git clones = separate workspaces = separate daemons (correct). Git worktrees share .beads/ and have known limitations (documented, use --no-daemon).","design":"Use flock (file locking) on daemon socket or database file to enforce singleton:\n\n1. On daemon start, attempt exclusive lock on .beads/bd.sock or .beads/daemon.lock\n2. If lock held by another process, refuse to start (exit with clear error)\n3. Hold lock for lifetime of daemon process\n4. Release lock on daemon shutdown\n\nAlternative: Use PID file with stale detection (check if PID is still running)\n\nImplementation location: Daemon startup code in cmd/bd/ or internal/daemon/","acceptance_criteria":"1. Starting second daemon process in same workspace fails with clear error\n2. Test: Start daemon, attempt second start, verify failure\n3. Killing daemon releases lock, allowing new daemon to start\n4. No infinite .beads/ directory recursion possible\n5. Works correctly with auto-start mechanism","status":"open","priority":0,"issue_type":"bug","created_at":"2025-10-25T22:23:59.433044-07:00","updated_at":"2025-10-25T22:23:59.433044-07:00"}
{"id":"bd-15","title":"Make merge command idempotent for safe retry after partial failures","description":"The merge command currently performs 3 operations without an outer transaction:\n1. Migrate dependencies from source → target\n2. Update text references across all issues\n3. Close source issues\n\nIf merge fails mid-operation (network issue, daemon crash, etc.), a retry will fail or produce incorrect results because some operations already succeeded.\n\n**Goal:** Make merge idempotent so retrying after partial failure is safe and completes the remaining work.\n\n**Idempotency checks needed:**\n- Skip dependency migration if target already has the dependency\n- Skip text reference updates if already updated\n- Skip closing source issues if already closed\n- Report which operations were skipped vs performed\n\n**Example output:**\n```\n✓ Merged 2 issue(s) into bd-63\n - Dependencies: 3 migrated, 2 already existed\n - Text references: 5 updated, 0 already correct\n - Source issues: 1 closed, 1 already closed\n```\n\n**Related:** bd-115 originally requested transaction support, but idempotency is a better solution for this use case since individual operations are already atomic.","design":"Current merge code already has some idempotency:\n- Dependency migration checks `alreadyExists` before adding (line ~145-151 in merge.go)\n- Text reference updates are naturally idempotent (replacing bd-X with bd-Y twice has same result)\n\nMissing idempotency:\n- CloseIssue fails if source already closed\n- Error messages don't distinguish \"already done\" from \"real failure\"\n\nImplementation:\n1. Check source issue status before closing - skip if already closed\n2. Track which operations succeeded/skipped\n3. Return detailed results for user visibility\n4. Consider adding --dry-run output showing what would be done vs skipped","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165434-07:00","updated_at":"2025-10-24T13:51:54.437619-07:00","closed_at":"2025-10-22T11:56:36.526276-07:00"}
{"id":"bd-150","title":"Implement exclusive lock protocol for daemon/external tool coexistence","description":"VC (VibeCoder) cannot coexist with bd daemon managing the same database. VC needs deterministic execution and directly manages JSONL sync, but daemon's auto-flush creates timing conflicts and nondeterministic behavior.\n\nCurrent workaround (vc-195): VC kills ALL bd daemon processes with pkill, which is too aggressive and prevents developers from using daemon for their own workflow.\n\nSolution: Implement lock file protocol (.beads/.exclusive-lock) that allows applications to claim exclusive management of a database. bd daemon respects these locks and skips locked databases.\n\nSee VC_DAEMON_EXCLUSION_PROTOCOL.md for full specification.","design":"Lock File Protocol:\n- File: .beads/.exclusive-lock (JSON format)\n- Contains: holder name, PID, hostname, timestamp, version\n- Created when external tool (VC) starts\n- Removed on clean shutdown\n- Stale lock detection: if PID doesn't exist, lock is removed\n\nbd daemon behavior:\n1. Before touching any database, check for .exclusive-lock\n2. If lock exists and valid (PID alive): skip database in sync cycle\n3. If lock is stale (PID dead): remove lock and proceed\n4. If lock is malformed: log warning and skip (fail-safe)\n\nExternal tool behavior:\n1. On startup: create .exclusive-lock with process info\n2. If lock exists, check if stale; fail if another process is running\n3. On shutdown: remove .exclusive-lock\n\nBenefits:\n- Clean separation, no process killing\n- Developer-friendly (can run daemon for other projects)\n- Fail-safe with stale lock detection\n- Extensible for other tools beyond VC","acceptance_criteria":"1. bd daemon skips databases with valid exclusive locks\n2. bd daemon cleans up stale locks automatically\n3. Lock file format documented and implemented\n4. Integration tests show daemon + external tool coexistence\n5. Documentation updated with protocol specification","status":"open","priority":1,"issue_type":"epic","created_at":"2025-10-25T22:43:08.994891-07:00","updated_at":"2025-10-25T22:43:08.994891-07:00"}
{"id":"bd-151","title":"Add ExclusiveLock struct and JSON marshaling","description":"Create Go struct to represent exclusive lock file format with JSON marshaling support.\n\nFields needed:\n- Holder (string): name of lock holder (e.g., \"vc-executor\")\n- PID (int): process ID\n- Hostname (string): hostname where process is running\n- StartedAt (time.Time): when lock was acquired\n- Version (string): version of lock holder\n\nShould support both Marshal and Unmarshal for reading/writing lock files.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:18.536901-07:00","updated_at":"2025-10-25T22:43:18.536901-07:00","dependencies":[{"issue_id":"bd-151","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:18.538184-07:00","created_by":"daemon"}]}
{"id":"bd-152","title":"Implement isProcessAlive() helper for PID validation","description":"Implement helper function to check if a process is alive given PID and hostname.\n\nLogic:\n- If hostname != current host: return true (can't verify remote, assume alive)\n- If hostname == current host: check if PID exists using os.FindProcess + Signal(0)\n- Return true if process exists, false otherwise\n\nNeeded for stale lock detection.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:26.074997-07:00","updated_at":"2025-10-25T22:43:26.074997-07:00","dependencies":[{"issue_id":"bd-152","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:28.313879-07:00","created_by":"daemon"}]}
{"id":"bd-153","title":"Implement shouldSkipDatabase() to check for exclusive locks","description":"Add shouldSkipDatabase() function to daemon that checks for .beads/.exclusive-lock before processing a database.\n\nLogic:\n1. Check if .beads/.exclusive-lock exists\n2. If not: return false (proceed with database)\n3. If yes: read and parse JSON\n4. If malformed: log warning and skip (fail-safe)\n5. If valid: check if PID is alive\n6. If stale (PID dead): remove lock file and proceed\n7. If active (PID alive): log skip message and return true\n\nDepends on ExclusiveLock struct and isProcessAlive() helper.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:34.561247-07:00","updated_at":"2025-10-25T22:43:34.561247-07:00","dependencies":[{"issue_id":"bd-153","depends_on_id":"bd-151","type":"blocks","created_at":"2025-10-25T22:43:36.6504-07:00","created_by":"daemon"},{"issue_id":"bd-153","depends_on_id":"bd-152","type":"blocks","created_at":"2025-10-25T22:43:38.730862-07:00","created_by":"daemon"},{"issue_id":"bd-153","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:40.602161-07:00","created_by":"daemon"}]}
{"id":"bd-154","title":"Integrate exclusive lock check into daemon sync cycle","description":"Integrate shouldSkipDatabase() into daemon's sync cycle to respect exclusive locks.\n\nBefore processing each database:\n1. Call shouldSkipDatabase(beadsDir)\n2. If true: skip all operations (export, import, commit, push)\n3. If false: proceed with normal sync\n\nAdd appropriate logging:\n- \"Skipping .beads/foo.db (locked by vc-executor, PID 12345)\" when skipping\n- \"Removed stale lock at .beads/foo.db (PID 12345 not found)\" when cleaning up\n\nDepends on shouldSkipDatabase() implementation.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:46.221736-07:00","updated_at":"2025-10-25T22:43:46.221736-07:00","dependencies":[{"issue_id":"bd-154","depends_on_id":"bd-153","type":"blocks","created_at":"2025-10-25T22:43:48.204682-07:00","created_by":"daemon"},{"issue_id":"bd-154","depends_on_id":"bd-150","type":"parent-child","created_at":"2025-10-25T22:43:50.283951-07:00","created_by":"daemon"}]}
{"id":"bd-155","title":"Add unit tests for exclusive lock detection","description":"Add unit tests for exclusive lock functionality:\n\nTest cases:\n1. No lock file exists -\u003e shouldSkipDatabase returns false\n2. Valid lock with alive PID -\u003e shouldSkipDatabase returns true\n3. Stale lock with dead PID -\u003e removes lock, returns false\n4. Malformed lock JSON -\u003e logs warning, returns true (fail-safe)\n5. Lock with different hostname -\u003e assumes alive, returns true\n6. isProcessAlive() correctly detects running/dead processes\n\nUse test helpers to create lock files with specific PIDs.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-25T22:43:57.616-07:00","updated_at":"2025-10-25T22:43:57.616-07:00"}
{"id":"bd-16","title":"Global daemon should warn/reject --auto-commit and --auto-push","description":"When user runs 'bd daemon --global --auto-commit', it's unclear which repo the daemon will commit to (especially after fixing bd-62 where global daemon won't open a DB).\n\nOptions:\n1. Warn and ignore the flags in global mode\n2. Error out with clear message\n\nLine 87-91 already checks autoPush, but should skip check entirely for global mode. Add user-friendly messaging about flag incompatibility.","status":"closed","priority":3,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165645-07:00","updated_at":"2025-10-24T13:51:54.437812-07:00","closed_at":"2025-10-17T23:04:30.223432-07:00"}
{"id":"bd-17","title":"Add cross-repo issue references (future enhancement)","description":"Support referencing issues across different beads repositories. Useful for tracking dependencies between separate projects.\n\nProposed syntax:\n- Local reference: bd-63 (current behavior)\n- Cross-repo by path: ~/src/other-project#bd-456\n- Cross-repo by workspace name: @project2:bd-789\n\nUse cases:\n1. Frontend project depends on backend API issue\n2. Shared library changes blocking multiple projects\n3. System administrator tracking work across machines\n4. Monorepo with separate beads databases per component\n\nImplementation challenges:\n- Storage layer needs to query external databases\n- Dependency resolution across repos\n- What if external repo not available?\n- How to handle in JSONL export/import?\n- Security: should repos be able to read others?\n\nDesign questions to resolve first:\n1. Read-only references vs full cross-repo dependencies?\n2. How to handle repo renames/moves?\n3. Absolute paths vs workspace names vs git remotes?\n4. Should bd-38 auto-discover related repos?\n\nRecommendation: \n- Gather user feedback first\n- Start with read-only references\n- Implement as plugin/extension?\n\nContext: This is mentioned in bd-38 as approach #2. Much more complex than daemon multi-repo approach. Only implement if there's strong user demand.\n\nPriority: Backlog (4) - wait for user feedback before designing","status":"closed","priority":4,"issue_type":"feature","created_at":"2025-10-22T00:47:43.165857-07:00","updated_at":"2025-10-24T13:51:54.438011-07:00","closed_at":"2025-10-20T22:00:31.966891-07:00"}
{"id":"bd-18","title":"Make beads reusable as a Go library for external projects like vc","description":"Currently beads is only usable as a CLI tool. We want to use beads as a library in other Go projects like ~/src/vc so they can programmatically manage issues without shelling out to the bd CLI.\n\nGoals:\n- Export public API from internal packages\n- Document Go package usage\n- Provide examples of programmatic usage\n- Ensure vc can import and use beads storage layer directly\n\nUse case: The vc project needs issue tracking and wants to use beads as an embedded library rather than as a separate CLI tool.","notes":"UnderlyingDB() method implemented and tested. Core functionality complete. Still needs documentation updates (bd-26) and lifecycle safety enhancements (bd-25).","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-22T12:27:30.35968-07:00","updated_at":"2025-10-24T13:51:54.438187-07:00","closed_at":"2025-10-22T19:46:09.362533-07:00"}