Commit Graph

30 Commits

Author SHA1 Message Date
Steve Yegge
18f81051f3 Fix race condition in client socket cleanup (bd-4owj)
- Re-check socket existence after lock check to avoid stale socket state
- If socket is initially missing but daemon lock is held, re-check socket to handle daemon startup race
- Add test TestTryConnectWithTimeout_SocketExistenceRecheck to verify fix

Fixes bd-4owj
2025-11-23 20:58:11 -08:00
Steve Yegge
d7f4189e3e feat: Add 'bd count' command for counting and grouping issues
Implements a new 'bd count' command that provides efficient issue counting
with filtering and grouping capabilities.

Features:
- Basic count: Returns total count of issues matching filters
- All filtering options from 'bd list' (status, priority, type, assignee, labels, dates, etc.)
- Grouping via --by-* flags: status, priority, type, assignee, label
- JSON output support for both simple and grouped counts
- Both daemon and direct mode support

Implementation:
- Added OpCount operation and CountArgs to RPC protocol
- Added Count() method to RPC client
- Implemented handleCount() server-side handler with optimized bulk label fetching
- Created cmd/bd/count.go with full CLI implementation

Performance optimization:
- Pre-fetches all labels in a single query when using --by-label to avoid N+1 queries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 19:21:05 -05:00
Steve Yegge
fe705befbe Add RPC diagnostics with BD_RPC_DEBUG env var
- Add BD_RPC_DEBUG=1 for lightweight timing logs to stderr
- Log socket path, socket exists check, dial timing, health check timing
- Improve daemon status message when lock not held
- Helps field triage of connection issues without verbose daemon logs
- Fixes bd-j7e2
2025-11-07 21:29:22 -08:00
Steve Yegge
a236558a7a Add client self-heal for stale daemon.pid
- When socket missing and lock free, automatically remove stale daemon.pid
- Prevents stale artifacts from accumulating after daemon crashes
- Includes comprehensive test coverage
- Fixes bd-1mzt

Amp-Thread-ID: https://ampcode.com/threads/T-3f606a8a-d591-4412-b994-ea790889a04d
Co-authored-by: Amp <amp@ampcode.com>
2025-11-07 21:21:24 -08:00
Steve Yegge
f6bdf7c641 Reduce RPC dial timeout from 2s to 200ms for fast-fail (bd-expt)
- Changed TryConnect default from 2s to 200ms
- Updated fallback timeout in TryConnectWithTimeout
- Complements bd-wgu4 lock probe to eliminate 5s delays
- Fixes GH#243 (5s delay when daemon socket missing)
- Health checks still use longer timeouts via explicit TryConnectWithTimeout calls
2025-11-07 21:12:45 -08:00
Steve Yegge
ba1b856acb Standardize daemon detection with tryDaemonLock probe (bd-wgu4)
- Extract lock checking to internal/lockfile package
- Add lock probe in RPC client before connection attempts
- Update daemon discovery to use lock probe
- Eliminates unnecessary connection attempts when socket missing

Closes bd-wgu4

Amp-Thread-ID: https://ampcode.com/threads/T-3b863f21-3af4-49d3-9214-477d904b80fe
Co-authored-by: Amp <amp@ampcode.com>
2025-11-07 21:02:38 -08:00
Markus Flür
e7f532db93 Implementing an RPC monitoring solution with a web-ui as implementation example. (#244)
* bd sync: 2025-10-30 12:12:27

* Working on frontend

* bd sync: 2025-11-06 16:55:55

* feat: finish bd monitor human viewer

* Merge conflicts resolved and added tests

* bd sync: 2025-11-06 17:23:41

* bd sync: 2025-11-06 17:34:52

* feat: Add reload button and multiselect status filter to monitor

- Changed status filter from single select to multiselect with 'Open' selected by default
- Added reload button with visual feedback (hover/active states)
- Updated filterIssues() to handle multiple selected statuses
- Added reloadData() function that reloads both stats and issues
- Improved responsive design for mobile devices
- Filter controls now use flexbox layout with better spacing

* fix: Update monitor statistics to show Total, In Progress, Open, Closed

- Replaced 'Ready to Work' stat with 'In Progress' stat
- Reordered stats to show logical progression: Total -> In Progress -> Open -> Closed
- Updated loadStats() to fetch in-progress count from stats API
- Removed unnecessary separate API call for ready count

* fix: Correct API field names in monitor stats JavaScript

The JavaScript was using incorrect field names (stats.total, stats.by_status)
that don't match the actual types.Statistics struct which uses flat fields
with underscores (total_issues, in_progress_issues, etc).

Fixed by updating loadStats() to use correct field names:
- stats.total -> stats.total_issues
- stats.by_status?.['in-progress'] -> stats.in_progress_issues
- stats.by_status?.open -> stats.open_issues
- stats.by_status?.closed -> stats.closed_issues

Fixes beads-9

* bd sync: 2025-11-06 17:51:24

* bd sync: 2025-11-06 17:56:09

* fix: Make monitor require daemon to prevent SQLite locking

Implemented Option 1 from beads-eel: monitor now requires daemon and never
opens direct SQLite connection.

Changes:
- Added 'monitor' to noDbCommands list in main.go to skip normal DB initialization
- Added validateDaemonForMonitor() PreRun function that:
  - Finds database path using beads.FindDatabasePath()
  - Validates daemon is running and healthy
  - Fails gracefully with clear error message if no daemon
  - Only uses RPC connection, never opens SQLite directly

Benefits:
- Eliminates SQLite locking conflicts between monitor and daemon
- Users can now close/update issues via CLI while monitor runs
- Clear error messages guide users to start daemon first

Fixes beads-eel

* bd sync: 2025-11-06 18:03:50

* docs: Add bd daemons restart subcommand documentation

Added documentation for the 'bd daemons restart' subcommand across all documentation files:

- commands/daemons.md: Added full restart subcommand section with synopsis, description, arguments, flags, and examples
- README.md: Added restart examples to daemon management section
- AGENTS.md: Added restart examples with --json flag for agents

The restart command gracefully stops and starts a specific daemon by workspace path or PID,
useful after upgrading bd or when a daemon needs refreshing.

Fixes beads-11

* bd sync: 2025-11-06 18:13:16

* Separated the web ui from the general monitoring functionality

---------

Co-authored-by: Steve Yegge <stevey@sourcegraph.com>
2025-11-07 09:49:12 -08:00
Steve Yegge
95cbcf4fbc Centralize BD_DEBUG logging into internal/debug package
- Created internal/debug package with Enabled(), Logf(), Printf()
- Added comprehensive unit tests for debug package
- Replaced 50+ scattered os.Getenv("BD_DEBUG") checks across 9 files
- Centralized debug logic for easier maintenance and testing
- All tests passing, behavior unchanged

Closes bd-fb95094c.5
2025-11-06 20:14:34 -08:00
Steve Yegge
cc7918daf4 Implement bd stale command (bd-c01f, closes #184)
- Add bd stale command to find abandoned/forgotten issues
- Support --days (default 30), --status, --limit, --json flags
- Implement GetStaleIssues in SQLite and Memory storage
- Add full RPC/daemon support
- Comprehensive test suite (6 tests, all passing)
- Update AGENTS.md documentation

Resolves GitHub issue #184

Amp-Thread-ID: https://ampcode.com/threads/T-f021ddb8-54e3-41bf-ba7a-071749663c1d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 23:03:56 -07:00
Steve Yegge
11c26d5af8 Add substring ID matching for all bd commands
- Enhanced ResolvePartialID to handle:
  - Bare hashes: 07b8c8 → bd-07b8c8
  - Prefix without hyphen: bd07b8c8 → bd-07b8c8
  - Full IDs: bd-07b8c8 (unchanged)
  - Substring matching: 07b → finds bd-07b8c8

- Added RPC support:
  - New OpResolveID operation
  - handleResolveID server handler
  - ResolveID client method

- Updated all commands to resolve IDs:
  - show, update, close, reopen
  - dep (add, remove, tree)
  - label (add, remove, list)

- Works in both daemon and direct modes

Fixes bd-0591c3
2025-10-30 19:20:50 -07:00
Steve Yegge
322ab63b10 Remove daemon storage cache (bd-33, bd-34, bd-35)
- Deleted server_cache_storage.go (~300 lines)
- Removed cache fields from Server struct
- Simplified database routing to use s.storage directly
- Removed cache metrics from health and metrics endpoints
- Deleted server_eviction_test.go (cache eviction tests)
- Cleaned up limits_test.go (removed cache assertions)
- All tests passing
2025-10-28 10:33:19 -07:00
Steve Yegge
d47378cfbc Remove skipped tests and unreachable RPC methods (bd-212, bd-213)
bd-212: Delete 3 permanently skipped test functions (~150 LOC)
- TestImportDependencyUpdates (import_collision_test.go)
- TestImportChainDependencies (import_collision_test.go)
- TestUpdateDependencyReferences (collision_test.go)

bd-213: Remove 4 unreachable RPC methods (~80 LOC)
- Server.GetLastImportTime
- Server.SetLastImportTime
- Server.findJSONLPath
- Client.Import

All tests pass. Phase 1 dead code cleanup continues.
2025-10-27 20:52:52 -07:00
Steve Yegge
93e170627d bd-154: Implement bd daemons stop and restart subcommands 2025-10-26 18:35:28 -07:00
Steve Yegge
75c959e69c feat(daemon): add GET /status endpoint (bd-148)
- Add OpStatus operation and StatusResponse type to RPC protocol
- Add workspacePath and dbPath fields to Server struct
- Implement handleStatus() handler with daemon metadata
- Track last activity time with atomic.Value
- Add client.Status() method
- Check for exclusive locks via ShouldSkipDatabase()
- Update all test files to use new NewServer signature
- Add comprehensive status endpoint test

Closes bd-148
2025-10-26 17:55:39 -07:00
Steve Yegge
0ee45bf7de fix: Auto-sync RPC client/server versions from CLI version
- Set rpc.ServerVersion from Version in daemon startup
- Set rpc.ClientVersion from Version in main.go startup
- Eliminates need to manually update RPC versions
- Both now use 0.0.0 placeholder, overridden at runtime

Amp-Thread-ID: https://ampcode.com/threads/T-03c37f7f-f41e-4b87-8700-d346c21bad30
Co-authored-by: Amp <amp@ampcode.com>
2025-10-26 14:18:35 -07:00
Steve Yegge
1a9258ce4f Remove deprecated bd repos command
Amp-Thread-ID: https://ampcode.com/threads/T-3fe46a3f-979f-48dd-9bb3-ee0b9fde46c2
Co-authored-by: Amp <amp@ampcode.com>
2025-10-24 19:49:16 -07:00
Steve Yegge
963181d7f8 Configure CI to pass lint checks for dependabot PRs
Disabled gocyclo and excluded baseline gosec warnings to allow CI to pass:
- Disabled gocyclo linter (high complexity in large functions is acceptable)
- Excluded test files from gosec checks (use dummy permissions/files)
- Excluded G204 (subprocess), G115 (int conversion), G302/G306 (file perms)
- Fixed unhandled errors: conn.Close(), rows.Close(), tempFile.Close()

Lint check now returns 0 issues (down from 56).

This fixes dependabot PR failures caused by lint checks.

Related: bd-91
2025-10-24 12:46:47 -07:00
Steve Yegge
9dcb86ebfb Fix lint errors: handle errors, use fmt.Fprintf, apply De Morgan's law, use switch statements
Amp-Thread-ID: https://ampcode.com/threads/T-afcf56b0-a8bc-4310-bb59-1b63e1d70c89
Co-authored-by: Amp <amp@ampcode.com>
2025-10-24 12:27:07 -07:00
Steve Yegge
0b819e1f40 feat: Add RPC support for epic commands in daemon mode
- Added OpEpicStatus operation to protocol
- Implemented handleEpicStatus() in RPC server
- Added EpicStatus() method to RPC client
- Updated epic.go to use daemon RPC when available
- Server-side filtering for close-eligible reduces RPC payload
- Both 'bd epic status' and 'bd epic close-eligible' now work in daemon mode

Fixes #62
Closes bd-87

Amp-Thread-ID: https://ampcode.com/threads/T-44c6044e-de04-40a0-bac3-b26238c32a17
Co-authored-by: Amp <amp@ampcode.com>
2025-10-23 22:04:13 -07:00
Steve Yegge
49ed3e74f0 Fix bd-2: Add daemon RPC support for sync export operations
- Added OpExport/OpImport to RPC protocol
- Implemented handleExport() in daemon to export via RPC
- Modified sync command to use daemon export when available
- Prevents nil pointer dereference when daemon is running
- Falls back to direct mode if daemon unavailable
2025-10-22 00:07:55 -07:00
Steve Yegge
645d268e43 Implement database handshake protocol in RPC layer
- Add ExpectedDB field to RPC Request
- Server validates client's expected DB matches daemon's DB
- Return clear error on mismatch with both paths
- Old clients (no ExpectedDB) still work with warning
- Add Path() method to storage.Storage interface
- Tests verify cross-database connections rejected

Prevents database pollution when client connects to wrong daemon.

Amp-Thread-ID: https://ampcode.com/threads/T-c4454192-39c6-4c67-96a9-675cbfc4db92
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 20:35:55 -07:00
Steve Yegge
34593cad8c Add RPC support for compact command (bd-184)
- Added OpCompact and OpCompactStats operation constants
- Added CompactArgs, CompactStatsArgs, and response types to RPC protocol
- Implemented handleCompact and handleCompactStats in RPC server
- Updated compact command to use RPC when daemon is available
- Fixed RPC client to include Cwd for proper database routing
- Compact now works in daemon mode with --no-daemon flag

Amp-Thread-ID: https://ampcode.com/threads/T-87885d07-80ad-466d-9ffb-cc96fab4853f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 00:29:50 -07:00
Steve Yegge
a86f3e139e Add native Windows support (#91)
- Native Windows daemon using TCP loopback endpoints
- Direct-mode fallback for CLI/daemon compatibility
- Comment operations over RPC
- PowerShell installer script
- Go 1.24 requirement
- Cross-OS testing documented

Co-authored-by: danshapiro <danshapiro@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-c6230265-055f-4af1-9712-4481061886db
Co-authored-by: Amp <amp@ampcode.com>
2025-10-20 21:08:49 -07:00
Steve Yegge
34cf361b2b Add telemetry and observability to daemon (bd-153)
Implement comprehensive metrics collection for the daemon with zero-overhead design:

Features:
- Request metrics: counts, latency percentiles (p50, p95, p99), error rates
- Cache metrics: hit/miss ratios, eviction counts, database connections
- Connection metrics: total, active, rejected connections
- System metrics: memory usage, goroutine count, uptime

Implementation:
- New internal/rpc/metrics.go with Metrics collector
- OpMetrics RPC operation for programmatic access
- 'bd daemon --metrics' command (human-readable and JSON output)
- Lock-free atomic operations for cache/connection metrics
- Copy-and-compute pattern in Snapshot to minimize lock contention
- Deferred metrics recording ensures all requests are tracked

Improvements from code review:
- JSON types use float64 for ms/seconds (not time.Duration)
- Snapshot copies data under short lock, computes outside
- Union of operations from counts and errors maps
- Defensive clamping in percentile calculation
- Defer pattern ensures metrics recorded even on early returns

Documentation updated in README.md with usage examples.

Closes bd-153

Amp-Thread-ID: https://ampcode.com/threads/T-20213187-65c7-47f7-ba21-5234c9e52e26
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 15:55:55 -07:00
Steve Yegge
22daa12665 Add daemon fallback visibility and version compatibility checks
Implemented bd-150: Improve daemon fallback visibility and user feedback
- Added DaemonStatus struct to track connection state
- Enhanced BD_DEBUG logging with detailed diagnostics and timing
- Added BD_VERBOSE mode with actionable warnings when falling back
- Implemented health checks before using daemon
- Clear fallback reasons: connect_failed, health_failed, auto_start_disabled, auto_start_failed, flag_no_daemon
- Updated documentation

Implemented bd-151: Add version compatibility checks for daemon RPC protocol
- Added ClientVersion field to RPC Request struct
- Client sends version (0.9.10) in all requests
- Server validates version compatibility using semver:
  - Major version must match
  - Daemon >= client for backward compatibility
  - Clear error messages with directional hints (upgrade daemon vs upgrade client)
- Added ClientVersion and Compatible fields to HealthResponse
- Implemented 'bd version --daemon' command to check compatibility
- Fixed batch operations to propagate ClientVersion for proper checks
- Updated documentation with version compatibility section

Code review improvements:
- Propagate ClientVersion in batch sub-requests
- Directional error messages based on which side is older
- Made ServerVersion a var for future unification

Amp-Thread-ID: https://ampcode.com/threads/T-b5fe36b8-c065-44a9-a55b-582573671609
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 08:04:48 -07:00
Steve Yegge
9e2ee1889f Add daemon health check endpoint (bd-146)
- Add OpHealth RPC operation to protocol
- Implement handleHealth() with DB ping and 1s timeout
- Returns status (healthy/degraded/unhealthy), uptime, cache metrics
- Update TryConnect() to use health check instead of ping
- Add 'bd daemon --health' CLI command with JSON output
- Track cache hits/misses for metrics
- Unhealthy daemon triggers automatic fallback to direct mode
- Health check completes in <2 seconds

Amp-Thread-ID: https://ampcode.com/threads/T-1a4889f3-77cf-433a-a704-e1c383929f48
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:41:06 -07:00
Steve Yegge
fb9b5864af feat: Add bd repos multi-repo commands and fix bd ready for in_progress issues
- Add 'bd repos' command for multi-repository management (bd-123)
  - bd repos list: show all cached repositories
  - bd repos ready: aggregate ready work across repos
  - bd repos stats: combined statistics across repos
  - bd repos clear-cache: clear repository cache
  - Requires global daemon (bd daemon --global)

- Fix bd ready to show in_progress issues (bd-165)
  - bd ready now shows both 'open' and 'in_progress' issues with no blockers
  - Allows epics/tasks ready to close to appear in ready work
  - Critical P0 bug fix for workflow

- Apply code review improvements to repos implementation
  - Use strongly typed RPC responses (remove interface{})
  - Fix clear-cache lock handling (close connections outside lock)
  - Add error collection for per-repo failures
  - Add context timeouts (1-2s) to prevent hangs
  - Add lock strategy comments

- Update documentation (README.md, AGENTS.md)
- Add comprehensive tests for both features

Amp-Thread-ID: https://ampcode.com/threads/T-1de989a1-1890-492c-9847-a34144259e0f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 00:37:27 -07:00
Steve Yegge
15b60b4ad0 Phase 4: Atomic operations and stress testing (bd-114, bd-110)
Completes daemon architecture implementation:

Features:
- Batch/transaction API (OpBatch) for multi-step atomic operations
- Request timeout and cancellation support (30s default, configurable)
- Comprehensive stress tests (4-10 concurrent agents, 800-1000 ops)
- Performance benchmarks (daemon 2x faster than direct mode)

Results:
- Zero ID collisions across 1000+ concurrent creates
- All acceptance criteria validated for bd-110
- Create: 2.4ms (daemon) vs 4.7ms (direct)
- Update/List: similar 2x improvement

Tests Added:
- TestStressConcurrentAgents (8 agents, 800 creates)
- TestStressBatchOperations (4 agents, 400 batch ops)
- TestStressMixedOperations (6 agents, mixed read/write)
- TestStressNoUniqueConstraintViolations (10 agents, 1000 creates)
- BenchmarkDaemonCreate/Update/List/Latency
- Fixed flaky TestConcurrentRequests (shared client issue)

Files:
- internal/rpc/protocol.go - Added OpBatch, BatchArgs, BatchResponse
- internal/rpc/server.go - Implemented handleBatch with stop-on-failure
- internal/rpc/client.go - Added SetTimeout and Batch methods
- internal/rpc/stress_test.go - All stress tests
- internal/rpc/bench_test.go - Performance benchmarks
- DAEMON_STRESS_TEST.md - Complete documentation

Closes bd-114, bd-110

Amp-Thread-ID: https://ampcode.com/threads/T-1c07c140-0420-49fe-add1-b0b83b1bdff5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 23:46:12 -07:00
Steve Yegge
5c0fac6e17 feat: Phase 1 RPC protocol infrastructure for daemon architecture (bd-111)
Implemented Unix socket RPC foundation to enable daemon-based concurrent access:

New files:
- internal/rpc/protocol.go: Request/Response types with 13 operations
- internal/rpc/server.go: Unix socket server with storage adapter
- internal/rpc/client.go: Client with auto-detection and typed methods
- internal/rpc/rpc_test.go: Integration tests

Features:
- JSON-based protocol over Unix sockets
- Adapter pattern for context/actor propagation to storage API
- Ping/health checks for daemon detection
- All core operations: create, update, close, list, show, ready, stats, deps, labels
- Graceful socket cleanup and signal handling
- Concurrent request support

Tests: 49.3% coverage, all passing

Related issues:
- bd-110: Daemon architecture epic
- bd-111: Phase 1 (completed)
- bd-112: Phase 2 (client auto-detection)
- bd-113: Phase 3 (daemon command)
- bd-114: Phase 4 (atomic operations)

Amp-Thread-ID: https://ampcode.com/threads/T-796c62e6-93b6-41c7-9cb5-8acc4a35ba9a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 22:49:19 -07:00
Steve Yegge
872f203c57 Add RPC infrastructure and updated database
- RPC Phase 1: Protocol, server, client implementation
- Updated renumber.go with proper text reference updates (3-phase approach)
- Clean database exported: 344 issues (bd-1 to bd-344)
- Added DAEMON_DESIGN.md documentation
- Updated go.mod/go.sum for RPC dependencies

Amp-Thread-ID: https://ampcode.com/threads/T-456af77c-8b7f-4004-9027-c37b95e10ea5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 20:36:23 -07:00