feat: Phase 1 RPC protocol infrastructure for daemon architecture (bd-111)

Implemented Unix socket RPC foundation to enable daemon-based concurrent access:

New files:
- internal/rpc/protocol.go: Request/Response types with 13 operations
- internal/rpc/server.go: Unix socket server with storage adapter
- internal/rpc/client.go: Client with auto-detection and typed methods
- internal/rpc/rpc_test.go: Integration tests

Features:
- JSON-based protocol over Unix sockets
- Adapter pattern for context/actor propagation to storage API
- Ping/health checks for daemon detection
- All core operations: create, update, close, list, show, ready, stats, deps, labels
- Graceful socket cleanup and signal handling
- Concurrent request support

Tests: 49.3% coverage, all passing

Related issues:
- bd-110: Daemon architecture epic
- bd-111: Phase 1 (completed)
- bd-112: Phase 2 (client auto-detection)
- bd-113: Phase 3 (daemon command)
- bd-114: Phase 4 (atomic operations)

Amp-Thread-ID: https://ampcode.com/threads/T-796c62e6-93b6-41c7-9cb5-8acc4a35ba9a
Co-authored-by: Amp <amp@ampcode.com>
This commit is contained in:
Steve Yegge
2025-10-16 22:49:19 -07:00
parent b87ef26b22
commit 5c0fac6e17
8 changed files with 1030 additions and 736 deletions

View File

@@ -10,6 +10,11 @@
{"id":"bd-107","title":"Make maxDepth configurable in bd dep tree command","description":"Currently maxDepth is hardcoded to 50 in GetDependencyTree. Add --max-depth flag to bd dep tree command to allow users to control recursion depth. Default should remain 50 for safety, but users with very deep trees or wanting shallow views should be able to configure it.","status":"open","priority":4,"issue_type":"feature","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-16T21:51:08.894613-07:00"}
{"id":"bd-108","title":"Fix renumbering temp ID collision bug","description":"bd renumber --force fails with UNIQUE constraint error when trying to assign temp IDs:\n\nError: failed to rename bd-4 to temp ID: failed to update issue ID: constraint failed: UNIQUE constraint failed: issues.id (1555)\n\nThe temp ID generation logic in renumber.go doesn't guarantee unique IDs. Need to:\n1. Use a temp ID strategy that can't collide (e.g., prefix with 'temp-', use UUIDs, or use high numbers like 999999+)\n2. Verify temp IDs don't exist before using them\n3. Add test case for renumbering with various ID gaps\n\nReproduced when renumbering 107 issues with gaps (IDs 1-344 compacting to 1-107).","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-10-16T21:13:38.519915-07:00","updated_at":"2025-10-16T21:51:08.895252-07:00","closed_at":"2025-10-16T21:19:18.49592-07:00"}
{"id":"bd-11","title":"Document or automate JSONL sync workflow for git collaboration","description":"When using beads across multiple machines/environments via git, there's a workflow gap:\n\n1. Machine A: Create issues → stored in .beads/project.db\n2. Machine A: bd export -o .beads/issues.jsonl\n3. Machine A: git add .beads/issues.jsonl \u0026\u0026 git commit \u0026\u0026 git push\n4. Machine B: git pull\n5. Machine B: ??? issues.jsonl exists but project.db is empty/stale\n\nThe missing step is: bd import --db .beads/project.db -i .beads/issues.jsonl\n\nThis needs to be either:\na) Documented clearly in workflow docs\nb) Automated (e.g., git hook, or bd auto-imports if jsonl is newer than db)\nc) Both\n\nReal-world impact: User had Claude Code on GCP VM create vc issues from BOOTSTRAP.md. They were exported to issues.jsonl and committed. But on local machine, vc.db was empty until manual import was run.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-16T21:51:08.743025-07:00","closed_at":"2025-10-14T02:51:52.199766-07:00"}
{"id":"bd-110","title":"Implement daemon architecture for concurrent access","description":"Multiple AI agents running concurrently cause database corruption, git lock contention, and data loss. Implement a daemon-based architecture where bd daemon owns SQLite (single writer) and all bd commands become RPC clients when daemon is running. Batches git operations to prevent index.lock contention. Maintains backward compatibility with graceful fallback to direct mode. See DAEMON_DESIGN.md for full details.","design":"Architecture: Unix socket RPC with JSON payloads. bd commands auto-detect daemon socket, fall back to direct mode if not present. Daemon serializes all SQLite writes and batches git exports every 5 seconds. Per-repo daemon using .beads/bd.sock location.\n\nImplementation phases:\n1. RPC protocol infrastructure (protocol.go, server.go, client.go)\n2. Client auto-detection and fallback\n3. Daemon SQLite ownership and git batching\n4. Atomic operations and transactions","acceptance_criteria":"- 4 concurrent agents can run without errors\n- No UNIQUE constraint failures on ID generation\n- No git index.lock errors \n- SQLite counter stays in sync with actual issues\n- Graceful fallback when daemon not running\n- All existing tests pass\n- Documentation updated","status":"open","priority":0,"issue_type":"epic","created_at":"2025-10-16T21:54:48.794119-07:00","updated_at":"2025-10-16T21:54:48.794119-07:00","dependencies":[{"issue_id":"bd-110","depends_on_id":"bd-111","type":"parent-child","created_at":"2025-10-16T21:54:56.032869-07:00","created_by":"stevey"}]}
{"id":"bd-111","title":"Phase 1: Implement RPC protocol infrastructure","description":"Create the foundation for daemon-client communication using Unix sockets and JSON.\n\nNew files to create:\n- internal/rpc/protocol.go - Request/response types, operations enum\n- internal/rpc/server.go - Unix socket server that daemon runs\n- internal/rpc/client.go - Client library for bd commands to use\n\nSocket location: .beads/bd.sock (per-repo)\n\nOperations to support initially: create, update, list, show, close, ready, stats","design":"protocol.go defines:\n- Request struct with Operation string and Args json.RawMessage\n- Response struct with Success bool, Data json.RawMessage, Error string\n- Operation constants for all bd commands\n\nserver.go implements:\n- Unix socket listener on .beads/bd.sock\n- Request handler that dispatches to storage layer\n- Graceful shutdown on signals\n\nclient.go implements:\n- TryConnect() to detect running daemon\n- Execute(operation, args) to send RPC request\n- Connection pooling/reuse for performance","acceptance_criteria":"- internal/rpc package compiles without errors\n- Server can accept and respond to simple ping request\n- Client can connect to socket and receive response\n- Unit tests for protocol serialization/deserialization\n- Socket cleanup on server shutdown","status":"closed","priority":0,"issue_type":"task","created_at":"2025-10-16T21:54:48.83081-07:00","updated_at":"2025-10-16T22:02:40.675096-07:00","closed_at":"2025-10-16T22:02:40.675096-07:00"}
{"id":"bd-112","title":"Phase 2: Add client auto-detection in bd commands","description":"Modify all bd commands to detect if daemon is running and route through RPC client if available, otherwise fall back to direct storage access.\n\nChanges needed:\n- Update cmd/bd/main.go to check for daemon socket on startup\n- Wrap storage calls with TryConnect logic\n- Ensure all commands work identically in both modes\n- Add --no-daemon flag to force direct mode\n\nThis maintains backward compatibility while enabling daemon mode.","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-16T22:47:36.185502-07:00","updated_at":"2025-10-16T22:47:36.185502-07:00","dependencies":[{"issue_id":"bd-112","depends_on_id":"bd-110","type":"parent-child","created_at":"2025-10-16T22:47:36.190931-07:00","created_by":"stevey"}]}
{"id":"bd-113","title":"Phase 3: Implement daemon command with SQLite ownership","description":"Create 'bd daemon' command that starts the RPC server and owns the SQLite database.\n\nImplementation:\n- Add cmd/bd/daemon.go with start/stop/status subcommands\n- Daemon holds exclusive SQLite connection\n- Integrates git sync loop (batch exports every 5 seconds)\n- PID file management for daemon lifecycle\n- Logging for daemon operations\n\nSocket location: .beads/bd.sock per repository","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-16T22:47:42.86546-07:00","updated_at":"2025-10-16T22:47:42.86546-07:00","dependencies":[{"issue_id":"bd-113","depends_on_id":"bd-110","type":"parent-child","created_at":"2025-10-16T22:47:42.874284-07:00","created_by":"stevey"}]}
{"id":"bd-114","title":"Phase 4: Add atomic operations and stress testing","description":"Implement atomic multi-operation support and test under concurrent load.\n\nFeatures:\n- Batch/transaction API for multi-step operations\n- Request timeout and cancellation support\n- Connection pooling optimization\n- Stress tests with 4+ concurrent agents\n- Performance benchmarks vs direct mode\n- Documentation updates\n\nValidates all acceptance criteria for bd-110.","status":"open","priority":0,"issue_type":"task","created_at":"2025-10-16T22:47:49.785525-07:00","updated_at":"2025-10-16T22:47:49.785525-07:00","dependencies":[{"issue_id":"bd-114","depends_on_id":"bd-110","type":"parent-child","created_at":"2025-10-16T22:47:49.787472-07:00","created_by":"stevey"}]}
{"id":"bd-12","title":"Root issue for dep tree test","description":"","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-16T21:51:08.743864-07:00","closed_at":"2025-10-16T10:07:34.1266-07:00"}
{"id":"bd-13","title":"Dependency A","description":"","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-16T21:51:08.74444-07:00","closed_at":"2025-10-16T10:07:34.126732-07:00"}
{"id":"bd-14","title":"Dependency B","description":"","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-16T21:51:08.745041-07:00","closed_at":"2025-10-16T10:07:34.126858-07:00"}