Commit Graph

47 Commits

Author SHA1 Message Date
Steve Yegge
ac1752d87d Complete cache audit (bd-bc2c6191)
- Created CACHE_AUDIT.md with comprehensive findings
- Confirmed cache was already removed in commit 322ab63
- Fixed stale comment in internal/rpc/server.go
- All tests passing, MCP multi-repo working correctly
- Closed bd-bc2c6191

Amp-Thread-ID: https://ampcode.com/threads/T-c1286278-b1ff-4b8a-b090-2b3a1c38c9dd
Co-authored-by: Amp <amp@ampcode.com>
2025-11-06 19:51:16 -08:00
Steve Yegge
0c737025b5 Split internal/rpc/server.go into 8 focused modules (bd-215)
Refactored monolithic 2238-line server.go into 8 files with clear responsibilities:
- server_core.go: Server type, NewServer (115 lines)
- server_lifecycle_conn.go: Start/Stop/connection handling (248 lines)
- server_cache_storage.go: Storage caching and eviction (286 lines)
- server_routing_validation_diagnostics.go: Request routing/validation (384 lines)
- server_issues_epics.go: Issue CRUD operations (506 lines)
- server_labels_deps_comments.go: Labels/deps/comments (199 lines)
- server_compact.go: Compaction operations (287 lines)
- server_export_import_auto.go: Export/import operations (293 lines)

Improvements:
- Replaced RWMutex.TryLock with atomic.Bool for portable single-flight guard
- Added default storage close in Stop() to prevent FD leaks
- All methods remain on *Server receiver (no behavior changes)
- Each file <510 LOC for better maintainability
- All tests pass, daemon verified working

Amp-Thread-ID: https://ampcode.com/threads/T-92d481ad-1bda-4ecd-bcf5-874a1889db30
Co-authored-by: Amp <amp@ampcode.com>
2025-10-27 21:14:34 -07:00
Steve Yegge
d47378cfbc Remove skipped tests and unreachable RPC methods (bd-212, bd-213)
bd-212: Delete 3 permanently skipped test functions (~150 LOC)
- TestImportDependencyUpdates (import_collision_test.go)
- TestImportChainDependencies (import_collision_test.go)
- TestUpdateDependencyReferences (collision_test.go)

bd-213: Remove 4 unreachable RPC methods (~80 LOC)
- Server.GetLastImportTime
- Server.SetLastImportTime
- Server.findJSONLPath
- Client.Import

All tests pass. Phase 1 dead code cleanup continues.
2025-10-27 20:52:52 -07:00
Steve Yegge
adfe177dba Fix bd-132: Implement daemon auto-import after git pull
- Created internal/importer package with all import logic
- Moved import phases from cmd/bd to internal/importer
- Implemented real importFunc in daemon's checkAndAutoImportIfStale()
- Added single-flight concurrency guard to prevent parallel imports
- Added fast mtime check to avoid unnecessary file reads (99% of requests <0.1ms)
- Fixed import options: RenameOnImport=true instead of SkipPrefixValidation
- Added export trigger after ID remapping to prevent collision loops
- Fixed memory storage interface: added GetDirtyIssueHash, GetExportHash, SetExportHash
- Updated GetDependencyTree signature for reverse parameter

Performance:
- Mtime check: ~0.01ms per request
- Import when needed: ~10-100ms (rare, only after git pull)
- Throughput maintained: 4300+ issues/sec
- No duplicate work with single-flight guard

Fixes critical data corruption bug where daemon served stale data after
git pull, causing fresh JSONL changes to be overwritten.

Amp-Thread-ID: https://ampcode.com/threads/T-71224a2d-b2d7-4173-b21e-449b64f9dd71
Co-authored-by: Amp <amp@ampcode.com>
2025-10-27 16:29:12 -07:00
Steve Yegge
648ecfafe7 Address gosec security warnings (bd-102)
- Enable gosec linter in .golangci.yml
- Tighten file permissions: 0755→0750 for directories, 0644→0600 for configs
- Git hooks remain 0700 (executable, user-only access)
- Add #nosec comments for safe cases with justifications:
  - G204: Safe subprocess launches (git show, bd daemon)
  - G304: File inclusions with controlled paths
  - G201: SQL formatting with controlled column names
  - G115: Integer conversions with controlled values

All gosec warnings resolved (20→0). All tests passing.

Amp-Thread-ID: https://ampcode.com/threads/T-d7166b9e-cbbe-4c7b-9e48-3df36b20f0d0
Co-authored-by: Amp <amp@ampcode.com>
2025-10-26 22:48:19 -07:00
Steve Yegge
93e170627d bd-154: Implement bd daemons stop and restart subcommands 2025-10-26 18:35:28 -07:00
Steve Yegge
75c959e69c feat(daemon): add GET /status endpoint (bd-148)
- Add OpStatus operation and StatusResponse type to RPC protocol
- Add workspacePath and dbPath fields to Server struct
- Implement handleStatus() handler with daemon metadata
- Track last activity time with atomic.Value
- Add client.Status() method
- Check for exclusive locks via ShouldSkipDatabase()
- Update all test files to use new NewServer signature
- Add comprehensive status endpoint test

Closes bd-148
2025-10-26 17:55:39 -07:00
Steve Yegge
0ee45bf7de fix: Auto-sync RPC client/server versions from CLI version
- Set rpc.ServerVersion from Version in daemon startup
- Set rpc.ClientVersion from Version in main.go startup
- Eliminates need to manually update RPC versions
- Both now use 0.0.0 placeholder, overridden at runtime

Amp-Thread-ID: https://ampcode.com/threads/T-03c37f7f-f41e-4b87-8700-d346c21bad30
Co-authored-by: Amp <amp@ampcode.com>
2025-10-26 14:18:35 -07:00
Steve Yegge
ada7bd0b73 Refactor autoImportIfNewer to internal/autoimport package (bd-128)
- Extracted auto-import logic from cmd/bd/main.go to internal/autoimport
- Removed global dependencies (store, dbPath) by using parameters
- Uses callback pattern (ImportFunc) for flexible import implementation
- Both CLI and daemon can now call auto-import after detecting staleness
- Added detailed ID remapping output for collision resolution
- Improved error reporting for parse failures
- All tests passing

Amp-Thread-ID: https://ampcode.com/threads/T-b7faaa33-fc52-409f-82b3-28db143b335d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-26 11:55:24 -07:00
Steve Yegge
daa25db720 Implement timestamp tracking and staleness detection (bd-159, bd-160)
- bd-159: Track last_import_time in metadata after auto-import
- bd-160: Add staleness check to daemon before serving requests
  - Detects when JSONL mtime > last import time
  - Currently logs warning (needs bd-166 for actual import trigger)
- Add lastImportTime field to RPC Server struct with getter/setter
- Add checkAndAutoImportIfStale() method to detect stale JSONL

This is part of bd-158 epic to fix daemon showing stale data after git pull.

Amp-Thread-ID: https://ampcode.com/threads/T-b554e049-aff8-4a24-8bf3-3305483b7f5a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-25 23:07:30 -07:00
Steve Yegge
09c11a26e6 Add configurable sort policy for GetReadyWork (bd-147)
- Add SortPolicy type with hybrid, priority, oldest constants
- Add SortPolicy field to WorkFilter
- Implement buildOrderByClause() for SQL generation
- Add --sort flag to bd ready command
- Add comprehensive tests for all 3 sort policies
- Update RPC protocol to support sort policy
- Update documentation with sort policy examples

Enables autonomous systems like VC to use strict priority ordering
while preserving hybrid behavior for interactive use.

Amp-Thread-ID: https://ampcode.com/threads/T-9d7ea9db-8d6d-4498-9daa-48a7e104ce1f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-25 18:54:00 -07:00
Steve Yegge
b855c444d4 Enable errcheck linter and fix all production code warnings
- Enabled errcheck linter (previously disabled)
- Set tests: false in .golangci.yml to focus on production code
- Fixed 27 errcheck warnings using Go best practices:
  * Database resources: defer func() { _ = rows.Close() }()
  * Transaction rollbacks: defer func() { _ = tx.Rollback() }()
  * Best-effort closers: _ = store.Close(), _ = client.Close()
  * File writes: proper error checking on Close()
  * Interactive input: handle EOF gracefully
  * File ops: ignore ENOENT on os.Remove()
- All tests pass
- Closes bd-58

Amp-Thread-ID: https://ampcode.com/threads/T-57c9afd3-9adf-40c2-8be7-3e493d200361
Co-authored-by: Amp <amp@ampcode.com>
2025-10-25 18:44:38 -07:00
Steve Yegge
bb33007036 Fix revive style issues (bd-56)
- Fix 14 unused-parameter warnings (rename to _)
- Fix 2 redefines-builtin-id (max→maxCount, min→minInt)
- Fix 3 indent-error-flow issues with gofmt
- Merged duplicate bd-126 into bd-116
2025-10-25 18:13:49 -07:00
Steve Yegge
1a9258ce4f Remove deprecated bd repos command
Amp-Thread-ID: https://ampcode.com/threads/T-3fe46a3f-979f-48dd-9bb3-ee0b9fde46c2
Co-authored-by: Amp <amp@ampcode.com>
2025-10-24 19:49:16 -07:00
Steve Yegge
963181d7f8 Configure CI to pass lint checks for dependabot PRs
Disabled gocyclo and excluded baseline gosec warnings to allow CI to pass:
- Disabled gocyclo linter (high complexity in large functions is acceptable)
- Excluded test files from gosec checks (use dummy permissions/files)
- Excluded G204 (subprocess), G115 (int conversion), G302/G306 (file perms)
- Fixed unhandled errors: conn.Close(), rows.Close(), tempFile.Close()

Lint check now returns 0 issues (down from 56).

This fixes dependabot PR failures caused by lint checks.

Related: bd-91
2025-10-24 12:46:47 -07:00
Steve Yegge
c2c7eda14f Fix 15 lint errors: dupl, gosec, revive, staticcheck, unparam
Reduced golangci-lint issues from 56 to 41:

Fixed:
- dupl (2→0): Extracted parseLabelArgs helper, added nolint for cobra commands
- gosec G104 (4→0): Handle unhandled errors with _ = assignments
- gosec G302/G306 (4→0): Fixed file permissions from 0644 to 0600
- revive exported (4→0): Added proper godoc comments for all exported types
- staticcheck SA1019 (1→0): Removed deprecated netErr.Temporary() call
- staticcheck SA4003 (1→0): Removed impossible uint64 < 0 check
- unparam (8→0): Removed unused params/returns, added nolint where needed

Renamed types in compact package to avoid stuttering:
- CompactConfig → Config
- CompactResult → Result

Remaining 41 issues are documented baseline:
- gocyclo (24): High complexity in large functions
- gosec G204/G115 (17): False positives for subprocess/conversions

Closes bd-92

Amp-Thread-ID: https://ampcode.com/threads/T-1c136506-d703-4781-bcfa-eb605999545a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-24 12:40:56 -07:00
Steve Yegge
9dcb86ebfb Fix lint errors: handle errors, use fmt.Fprintf, apply De Morgan's law, use switch statements
Amp-Thread-ID: https://ampcode.com/threads/T-afcf56b0-a8bc-4310-bb59-1b63e1d70c89
Co-authored-by: Amp <amp@ampcode.com>
2025-10-24 12:27:07 -07:00
Steve Yegge
c59db1a798 fix: Resolve 11 errcheck linter violations to unblock CI (bd-91)
Fixed all unchecked error returns in production code:
- os.Remove() calls in cleanup paths
- cmd.Wait() in goroutines
- fmt.Fprintf() writes
- Type assertions with proper ok checks

Reduces linter issues from 99 to 88. CI should now pass linting.
2025-10-24 11:59:11 -07:00
Steve Yegge
42480014b3 Additional Windows CI fixes (bd-99)
- Write PID to daemon.pid when acquiring lock for Windows compatibility
- Increase socket cleanup timeout to 5s for Windows
- Windows can't read locked files, so PID file is required fallback
2025-10-24 10:12:59 -07:00
Steve Yegge
09e51b2184 Fix Windows CI test failures (bd-99)
- Fix PID detection: Open lock file with O_RDWR for Windows LockFileEx
- Fix script tests: Increase timeout to 2-5s for Windows process startup
- Fix uptime test: Use math.Ceil with minimum 1 second to prevent flakiness
- Fix socket cleanup: Add done channel to wait for Start() cleanup completion

All 5 failing Windows tests should now pass.
2025-10-24 10:07:05 -07:00
Steve Yegge
e293974c71 Fix CI failures: errcheck lint errors and flaky memory pressure test 2025-10-24 00:27:07 -07:00
Steve Yegge
0b819e1f40 feat: Add RPC support for epic commands in daemon mode
- Added OpEpicStatus operation to protocol
- Implemented handleEpicStatus() in RPC server
- Added EpicStatus() method to RPC client
- Updated epic.go to use daemon RPC when available
- Server-side filtering for close-eligible reduces RPC payload
- Both 'bd epic status' and 'bd epic close-eligible' now work in daemon mode

Fixes #62
Closes bd-87

Amp-Thread-ID: https://ampcode.com/threads/T-44c6044e-de04-40a0-bac3-b26238c32a17
Co-authored-by: Amp <amp@ampcode.com>
2025-10-23 22:04:13 -07:00
Steve Yegge
4f560379f2 Add description parameter to bd update command and MCP server
Fixes #114 and #122 by adding --description/-d flag to bd update CLI
and description parameter to MCP update_issue tool.

Changes:
- CLI: Added --description flag to updateCmd
- RPC: Added Description field to UpdateArgs
- Daemon: Updated updatesFromArgs to handle description
- MCP: Added description to update_issue, UpdateIssueParams, and clients
- Storage: description already supported in allowedUpdateFields

Tested in both daemon and direct modes.
2025-10-23 11:00:19 -07:00
Steve Yegge
49ed3e74f0 Fix bd-2: Add daemon RPC support for sync export operations
- Added OpExport/OpImport to RPC protocol
- Implemented handleExport() in daemon to export via RPC
- Modified sync command to use daemon export when available
- Prevents nil pointer dereference when daemon is running
- Falls back to direct mode if daemon unavailable
2025-10-22 00:07:55 -07:00
Steve Yegge
399fc73913 Fix labels and dependencies not saved in daemon mode (fixes #101)
The daemon's handleCreate function was ignoring createArgs.Labels and
createArgs.Dependencies, causing labels and deps specified during issue
creation to be lost when using daemon mode.

Added label and dependency processing after issue creation to match
the direct mode behavior in cmd/bd/main.go.

Fixes: #101
Amp-Thread-ID: https://ampcode.com/threads/T-cc80f8d7-264c-446e-b34f-79c4a7713b77
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 22:05:01 -07:00
Steve Yegge
d4e7af99a1 Fix bd-49: Daemon detects external DB modifications via mtime check
- Add dbMtime field to StorageCacheEntry to track DB file modification time
- Check mtime on cache hits and evict stale entries if DB changed externally
- Close and reopen storage when external modifications detected
- Fixes issue where daemon served stale data after direct DB operations

Amp-Thread-ID: https://ampcode.com/threads/T-631d5cca-0b26-47cb-b633-118b788483cf
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 21:55:23 -07:00
Steve Yegge
645d268e43 Implement database handshake protocol in RPC layer
- Add ExpectedDB field to RPC Request
- Server validates client's expected DB matches daemon's DB
- Return clear error on mismatch with both paths
- Old clients (no ExpectedDB) still work with warning
- Add Path() method to storage.Storage interface
- Tests verify cross-database connections rejected

Prevents database pollution when client connects to wrong daemon.

Amp-Thread-ID: https://ampcode.com/threads/T-c4454192-39c6-4c67-96a9-675cbfc4db92
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 20:35:55 -07:00
Steve Yegge
7ae148cf2b Add --id flag to bd list for filtering by specific issue IDs
- Add --id flag accepting comma-separated IDs
- Implements ID filtering at CLI, RPC, and storage layers
- Normalizes IDs (trim, dedupe, remove empty) like labels
- Guards against excessive ID lists (max 1000)
- Works with other filters (status, priority, etc.)

Closes bd-200

Amp-Thread-ID: https://ampcode.com/threads/T-377464f2-1e7f-46f9-b23e-1e3cfd611061
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 19:53:27 -07:00
Steve Yegge
34593cad8c Add RPC support for compact command (bd-184)
- Added OpCompact and OpCompactStats operation constants
- Added CompactArgs, CompactStatsArgs, and response types to RPC protocol
- Implemented handleCompact and handleCompactStats in RPC server
- Updated compact command to use RPC when daemon is available
- Fixed RPC client to include Cwd for proper database routing
- Compact now works in daemon mode with --no-daemon flag

Amp-Thread-ID: https://ampcode.com/threads/T-87885d07-80ad-466d-9ffb-cc96fab4853f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-21 00:29:50 -07:00
Steve Yegge
a86f3e139e Add native Windows support (#91)
- Native Windows daemon using TCP loopback endpoints
- Direct-mode fallback for CLI/daemon compatibility
- Comment operations over RPC
- PowerShell installer script
- Go 1.24 requirement
- Cross-OS testing documented

Co-authored-by: danshapiro <danshapiro@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-c6230265-055f-4af1-9712-4481061886db
Co-authored-by: Amp <amp@ampcode.com>
2025-10-20 21:08:49 -07:00
Steve Yegge
422c102f46 Add label filtering to bd list with AND/OR semantics
- Add --label flag for AND filtering (must have ALL labels)
- Add --label-any flag for OR filtering (must have AT LEAST ONE label)
- Add normalizeLabels() helper to trim, dedupe, and clean inputs
- Fix RPC title filtering parity bug (forward via Query field)
- Add comprehensive tests for label filtering including combined AND+OR
- Update documentation in README and CHANGELOG
- Improve flag help text to clarify combined semantics

Closes bd-161
2025-10-19 23:03:02 -07:00
Steve Yegge
f1ec927a7c feat: Add label display to bd show and bd list
- Populate labels in bd list human-readable output (direct mode)
- Populate labels in bd list JSON output (direct and daemon modes)
- Populate labels in daemon RPC handlers (handleList, handleShow)
- handleShow now returns full IssueDetails with labels/deps/dependents
- Labels displayed with 'Labels: [label1, label2]' format

Closes bd-164

Amp-Thread-ID: https://ampcode.com/threads/T-30cd607d-c509-4a8c-9cac-c2aea2ad75c6
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 22:44:39 -07:00
Steve Yegge
84a5ef7bf8 Fix daemon socket ready race condition (bd-151)
Add WaitReady() channel to RPC server that signals when the socket is
listening and ready to accept connections. Previously daemon startup
waited a fixed 2 seconds which could fail if the server took longer.

Changes:
- Add readyChan to Server struct
- Signal ready after listener bind completes
- Update daemon startup to wait on WaitReady() channel
- Increase timeout to 5s with proper signaling

This fixes multi-repo daemon routing test failures where daemon would
start but not be ready to handle requests within the timeout window.

Amp-Thread-ID: https://ampcode.com/threads/T-675a2db5-b1b3-480d-a108-b003d8139d08
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 20:43:22 -07:00
Steve Yegge
34cf361b2b Add telemetry and observability to daemon (bd-153)
Implement comprehensive metrics collection for the daemon with zero-overhead design:

Features:
- Request metrics: counts, latency percentiles (p50, p95, p99), error rates
- Cache metrics: hit/miss ratios, eviction counts, database connections
- Connection metrics: total, active, rejected connections
- System metrics: memory usage, goroutine count, uptime

Implementation:
- New internal/rpc/metrics.go with Metrics collector
- OpMetrics RPC operation for programmatic access
- 'bd daemon --metrics' command (human-readable and JSON output)
- Lock-free atomic operations for cache/connection metrics
- Copy-and-compute pattern in Snapshot to minimize lock contention
- Deferred metrics recording ensures all requests are tracked

Improvements from code review:
- JSON types use float64 for ms/seconds (not time.Duration)
- Snapshot copies data under short lock, computes outside
- Union of operations from counts and errors maps
- Defensive clamping in percentile calculation
- Defer pattern ensures metrics recorded even on early returns

Documentation updated in README.md with usage examples.

Closes bd-153

Amp-Thread-ID: https://ampcode.com/threads/T-20213187-65c7-47f7-ba21-5234c9e52e26
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 15:55:55 -07:00
Steve Yegge
c71da4c267 Add resource limits to daemon (bd-152)
Implemented connection limiting, request timeouts, and memory pressure detection:

- Connection limiting with semaphore pattern (default 100 max connections)
- Request timeout enforcement on read/write (default 30s)
- Memory pressure detection with aggressive cache eviction (default 500MB threshold)
- Configurable via environment variables:
  - BEADS_DAEMON_MAX_CONNS
  - BEADS_DAEMON_REQUEST_TIMEOUT
  - BEADS_DAEMON_MEMORY_THRESHOLD_MB
- Health endpoint now exposes active/max connections and memory usage
- Comprehensive test coverage for all limits

This prevents resource exhaustion under heavy load or attack scenarios.

Amp-Thread-ID: https://ampcode.com/threads/T-44d1817a-3709-4f1d-a27a-78bb2fa4d3dc
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 13:22:23 -07:00
Steve Yegge
5aa7658433 Fix race condition in TestSocketCleanup by protecting listener access with mutex
Fixes bd-160

The race was between Start() writing s.listener and Stop() reading it.
Now all listener access is protected by the server mutex:
- Start() stores listener under lock after creation
- Accept loop reads listener under RLock
- Stop() closes listener under lock

All RPC tests now pass with -race flag.
2025-10-19 09:14:37 -07:00
Steve Yegge
22daa12665 Add daemon fallback visibility and version compatibility checks
Implemented bd-150: Improve daemon fallback visibility and user feedback
- Added DaemonStatus struct to track connection state
- Enhanced BD_DEBUG logging with detailed diagnostics and timing
- Added BD_VERBOSE mode with actionable warnings when falling back
- Implemented health checks before using daemon
- Clear fallback reasons: connect_failed, health_failed, auto_start_disabled, auto_start_failed, flag_no_daemon
- Updated documentation

Implemented bd-151: Add version compatibility checks for daemon RPC protocol
- Added ClientVersion field to RPC Request struct
- Client sends version (0.9.10) in all requests
- Server validates version compatibility using semver:
  - Major version must match
  - Daemon >= client for backward compatibility
  - Clear error messages with directional hints (upgrade daemon vs upgrade client)
- Added ClientVersion and Compatible fields to HealthResponse
- Implemented 'bd version --daemon' command to check compatibility
- Fixed batch operations to propagate ClientVersion for proper checks
- Updated documentation with version compatibility section

Code review improvements:
- Propagate ClientVersion in batch sub-requests
- Directional error messages based on which side is older
- Made ServerVersion a var for future unification

Amp-Thread-ID: https://ampcode.com/threads/T-b5fe36b8-c065-44a9-a55b-582573671609
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 08:04:48 -07:00
Steve Yegge
bb13f4634b Fix daemon crash recovery race conditions (bd-147)
Improvements based on oracle code review:
- Move socket cleanup AFTER lock acquisition (prevents unlinking live sockets)
- Add PID liveness check before removing stale socket
- Add stale lock detection with retry mechanism
- Tighten directory permissions to 0700 for security
- Improve socket readiness probing with shorter timeouts
- Make removeOldSocket() ignore ENOENT errors

Fixes race condition where socket could be removed during daemon startup window,
potentially orphaning a running daemon process.

Amp-Thread-ID: https://ampcode.com/threads/T-63542c60-b5b9-4a34-9f22-415d9d7e8223
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:59:06 -07:00
Steve Yegge
9e2ee1889f Add daemon health check endpoint (bd-146)
- Add OpHealth RPC operation to protocol
- Implement handleHealth() with DB ping and 1s timeout
- Returns status (healthy/degraded/unhealthy), uptime, cache metrics
- Update TryConnect() to use health check instead of ping
- Add 'bd daemon --health' CLI command with JSON output
- Track cache hits/misses for metrics
- Unhealthy daemon triggers automatic fallback to direct mode
- Health check completes in <2 seconds

Amp-Thread-ID: https://ampcode.com/threads/T-1a4889f3-77cf-433a-a704-e1c383929f48
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:41:06 -07:00
Steve Yegge
f987722f96 Code review fixes for cache eviction (bd-145)
Oracle-recommended improvements:

**Thread Safety:**
- Fix lastAccess race: updates now under Write lock
- Make Stop() idempotent with sync.Once
- Close stores synchronously (not in goroutine)

**Performance:**
- Replace O(n²) sort with sort.Slice (O(n log n))
- Enforce LRU immediately on insert (prevents FD spikes)

**Correctness:**
- Canonicalize cache key to repo root (not cwd)
  - Prevents duplicate connections for same repo
  - Multiple subdirs → single cache entry
- Validate env vars: TTL <= 0 falls back to default

**Tests (6 new edge cases):**
- Canonical key behavior across subdirectories
- Immediate LRU enforcement without periodic cleanup
- Invalid TTL handling
- Re-open after eviction
- Stop idempotency
- All tests pass with -race flag

This addresses potential data races, resource spikes, and duplicate
connections identified during code review.
2025-10-18 13:24:59 -07:00
Steve Yegge
259e994522 Add storage cache eviction policy to daemon (bd-145)
Implemented TTL-based and LRU cache eviction for daemon storage connections:

- Add StorageCacheEntry with lastAccess timestamp tracking
- Cleanup goroutine runs every 5 minutes to evict stale entries
- TTL-based eviction: remove entries idle >30min (configurable)
- LRU eviction: enforce max cache size (default: 50 repos)
- Configurable via BEADS_DAEMON_MAX_CACHE_SIZE and BEADS_DAEMON_CACHE_TTL
- Proper cleanup on server shutdown
- Update lastAccess on cache hits
- Comprehensive tests for eviction logic

Fixes memory leaks and file descriptor exhaustion for multi-repo users.

Amp-Thread-ID: https://ampcode.com/threads/T-1148d8b3-b8a8-45fc-af9c-b5be14c4834d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:17:07 -07:00
Steve Yegge
fb9b5864af feat: Add bd repos multi-repo commands and fix bd ready for in_progress issues
- Add 'bd repos' command for multi-repository management (bd-123)
  - bd repos list: show all cached repositories
  - bd repos ready: aggregate ready work across repos
  - bd repos stats: combined statistics across repos
  - bd repos clear-cache: clear repository cache
  - Requires global daemon (bd daemon --global)

- Fix bd ready to show in_progress issues (bd-165)
  - bd ready now shows both 'open' and 'in_progress' issues with no blockers
  - Allows epics/tasks ready to close to appear in ready work
  - Critical P0 bug fix for workflow

- Apply code review improvements to repos implementation
  - Use strongly typed RPC responses (remove interface{})
  - Fix clear-cache lock handling (close connections outside lock)
  - Add error collection for per-repo failures
  - Add context timeouts (1-2s) to prevent hangs
  - Add lock strategy comments

- Update documentation (README.md, AGENTS.md)
- Add comprehensive tests for both features

Amp-Thread-ID: https://ampcode.com/threads/T-1de989a1-1890-492c-9847-a34144259e0f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 00:37:27 -07:00
Steve Yegge
0dac4b9003 Fix global daemon implementation and improve security
- bd-159: Global daemon now runs in routing mode without opening DB
- bd-158: Set socket permissions to 0600 for security
- bd-160: Reject --auto-commit/--auto-push with --global
- bd-157: Verified stale socket cleanup (already working)
- bd-56: Closed as won't-do (cycle prevention is better)
- bd-73: Multi-repo support complete
2025-10-17 23:17:22 -07:00
Steve Yegge
b40de9bc41 Implement daemon RPC with per-request context routing (bd-115)
- Added per-request storage routing in daemon server
  - Server now supports Cwd field in requests for database discovery
  - Tree-walking to find .beads/*.db from any working directory
  - Storage caching for performance across requests

- Created Python daemon client (bd_daemon_client.py)
  - RPC over Unix socket communication
  - Implements full BdClientBase interface
  - Auto-discovery of daemon socket from working directory

- Refactored bd_client.py with abstract interface
  - BdClientBase abstract class for common interface
  - BdCliClient for CLI-based operations (renamed from BdClient)
  - create_bd_client() factory with daemon/CLI fallback
  - Backwards-compatible BdClient alias

Next: Update MCP server to use daemon client when available
2025-10-17 16:28:29 -07:00
Steve Yegge
15b60b4ad0 Phase 4: Atomic operations and stress testing (bd-114, bd-110)
Completes daemon architecture implementation:

Features:
- Batch/transaction API (OpBatch) for multi-step atomic operations
- Request timeout and cancellation support (30s default, configurable)
- Comprehensive stress tests (4-10 concurrent agents, 800-1000 ops)
- Performance benchmarks (daemon 2x faster than direct mode)

Results:
- Zero ID collisions across 1000+ concurrent creates
- All acceptance criteria validated for bd-110
- Create: 2.4ms (daemon) vs 4.7ms (direct)
- Update/List: similar 2x improvement

Tests Added:
- TestStressConcurrentAgents (8 agents, 800 creates)
- TestStressBatchOperations (4 agents, 400 batch ops)
- TestStressMixedOperations (6 agents, mixed read/write)
- TestStressNoUniqueConstraintViolations (10 agents, 1000 creates)
- BenchmarkDaemonCreate/Update/List/Latency
- Fixed flaky TestConcurrentRequests (shared client issue)

Files:
- internal/rpc/protocol.go - Added OpBatch, BatchArgs, BatchResponse
- internal/rpc/server.go - Implemented handleBatch with stop-on-failure
- internal/rpc/client.go - Added SetTimeout and Batch methods
- internal/rpc/stress_test.go - All stress tests
- internal/rpc/bench_test.go - Performance benchmarks
- DAEMON_STRESS_TEST.md - Complete documentation

Closes bd-114, bd-110

Amp-Thread-ID: https://ampcode.com/threads/T-1c07c140-0420-49fe-add1-b0b83b1bdff5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 23:46:12 -07:00
Steve Yegge
5c0fac6e17 feat: Phase 1 RPC protocol infrastructure for daemon architecture (bd-111)
Implemented Unix socket RPC foundation to enable daemon-based concurrent access:

New files:
- internal/rpc/protocol.go: Request/Response types with 13 operations
- internal/rpc/server.go: Unix socket server with storage adapter
- internal/rpc/client.go: Client with auto-detection and typed methods
- internal/rpc/rpc_test.go: Integration tests

Features:
- JSON-based protocol over Unix sockets
- Adapter pattern for context/actor propagation to storage API
- Ping/health checks for daemon detection
- All core operations: create, update, close, list, show, ready, stats, deps, labels
- Graceful socket cleanup and signal handling
- Concurrent request support

Tests: 49.3% coverage, all passing

Related issues:
- bd-110: Daemon architecture epic
- bd-111: Phase 1 (completed)
- bd-112: Phase 2 (client auto-detection)
- bd-113: Phase 3 (daemon command)
- bd-114: Phase 4 (atomic operations)

Amp-Thread-ID: https://ampcode.com/threads/T-796c62e6-93b6-41c7-9cb5-8acc4a35ba9a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 22:49:19 -07:00
Steve Yegge
872f203c57 Add RPC infrastructure and updated database
- RPC Phase 1: Protocol, server, client implementation
- Updated renumber.go with proper text reference updates (3-phase approach)
- Clean database exported: 344 issues (bd-1 to bd-344)
- Added DAEMON_DESIGN.md documentation
- Updated go.mod/go.sum for RPC dependencies

Amp-Thread-ID: https://ampcode.com/threads/T-456af77c-8b7f-4004-9027-c37b95e10ea5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 20:36:23 -07:00