Commit Graph

91 Commits

Author SHA1 Message Date
Steve Yegge
34cf361b2b Add telemetry and observability to daemon (bd-153)
Implement comprehensive metrics collection for the daemon with zero-overhead design:

Features:
- Request metrics: counts, latency percentiles (p50, p95, p99), error rates
- Cache metrics: hit/miss ratios, eviction counts, database connections
- Connection metrics: total, active, rejected connections
- System metrics: memory usage, goroutine count, uptime

Implementation:
- New internal/rpc/metrics.go with Metrics collector
- OpMetrics RPC operation for programmatic access
- 'bd daemon --metrics' command (human-readable and JSON output)
- Lock-free atomic operations for cache/connection metrics
- Copy-and-compute pattern in Snapshot to minimize lock contention
- Deferred metrics recording ensures all requests are tracked

Improvements from code review:
- JSON types use float64 for ms/seconds (not time.Duration)
- Snapshot copies data under short lock, computes outside
- Union of operations from counts and errors maps
- Defensive clamping in percentile calculation
- Defer pattern ensures metrics recorded even on early returns

Documentation updated in README.md with usage examples.

Closes bd-153

Amp-Thread-ID: https://ampcode.com/threads/T-20213187-65c7-47f7-ba21-5234c9e52e26
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 15:55:55 -07:00
Steve Yegge
7d695f0b87 Fix SQL timestamp scanning error on macOS (bd-161, GH-88)
Fixes timestamp scanning error reported in GH-88 where DATETIME columns
were being returned as strings instead of time.Time on macOS 13.5.

Root cause: modernc.org/sqlite driver doesn't recognize mattn-style DSN
parameters (_journal_mode, _foreign_keys). When these incompatible
parameters are present, the driver ignores _time_format=sqlite on some
platforms, causing DATETIME values to remain as strings.

Solution: Use modernc's native _pragma syntax for all database options:
- Changed _journal_mode=WAL to _pragma=journal_mode(WAL)
- Changed _foreign_keys=ON to _pragma=foreign_keys(ON)
- Kept _pragma=busy_timeout(30000) and _time_format=sqlite

This ensures all parameters are properly recognized and DATETIME columns
are automatically parsed to time.Time across all platforms.

Fixes #88

Amp-Thread-ID: https://ampcode.com/threads/T-44d1817a-3709-4f1d-a27a-78bb2fa4d3dc
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 14:36:05 -07:00
Steve Yegge
c71da4c267 Add resource limits to daemon (bd-152)
Implemented connection limiting, request timeouts, and memory pressure detection:

- Connection limiting with semaphore pattern (default 100 max connections)
- Request timeout enforcement on read/write (default 30s)
- Memory pressure detection with aggressive cache eviction (default 500MB threshold)
- Configurable via environment variables:
  - BEADS_DAEMON_MAX_CONNS
  - BEADS_DAEMON_REQUEST_TIMEOUT
  - BEADS_DAEMON_MEMORY_THRESHOLD_MB
- Health endpoint now exposes active/max connections and memory usage
- Comprehensive test coverage for all limits

This prevents resource exhaustion under heavy load or attack scenarios.

Amp-Thread-ID: https://ampcode.com/threads/T-44d1817a-3709-4f1d-a27a-78bb2fa4d3dc
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 13:22:23 -07:00
Steve Yegge
5aa7658433 Fix race condition in TestSocketCleanup by protecting listener access with mutex
Fixes bd-160

The race was between Start() writing s.listener and Stop() reading it.
Now all listener access is protected by the server mutex:
- Start() stores listener under lock after creation
- Accept loop reads listener under RLock
- Stop() closes listener under lock

All RPC tests now pass with -race flag.
2025-10-19 09:14:37 -07:00
Steve Yegge
b0fba2eef2 feat: implement --max-depth flag for bd dep tree (closes #87, bd-3, bd-159)
- Add --max-depth/-d flag with default of 50
- Wire flag through to store.GetDependencyTree()
- Add input validation (must be >= 1)
- Show inline '… [truncated]' markers on truncated nodes
- Update truncation warning to show actual depth used
- Add comprehensive tests (truncation, default depth, boundary cases)
- Update CLI docs and reference

Thanks to @yashwanth-reddy909 for the initial implementation in PR #87.
This commit completes the feature with full wiring, validation, tests, and docs.

Amp-Thread-ID: https://ampcode.com/threads/T-c439b09c-cff2-48d9-8988-cf9353f0d32e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 09:00:11 -07:00
Steve Yegge
22daa12665 Add daemon fallback visibility and version compatibility checks
Implemented bd-150: Improve daemon fallback visibility and user feedback
- Added DaemonStatus struct to track connection state
- Enhanced BD_DEBUG logging with detailed diagnostics and timing
- Added BD_VERBOSE mode with actionable warnings when falling back
- Implemented health checks before using daemon
- Clear fallback reasons: connect_failed, health_failed, auto_start_disabled, auto_start_failed, flag_no_daemon
- Updated documentation

Implemented bd-151: Add version compatibility checks for daemon RPC protocol
- Added ClientVersion field to RPC Request struct
- Client sends version (0.9.10) in all requests
- Server validates version compatibility using semver:
  - Major version must match
  - Daemon >= client for backward compatibility
  - Clear error messages with directional hints (upgrade daemon vs upgrade client)
- Added ClientVersion and Compatible fields to HealthResponse
- Implemented 'bd version --daemon' command to check compatibility
- Fixed batch operations to propagate ClientVersion for proper checks
- Updated documentation with version compatibility section

Code review improvements:
- Propagate ClientVersion in batch sub-requests
- Directional error messages based on which side is older
- Made ServerVersion a var for future unification

Amp-Thread-ID: https://ampcode.com/threads/T-b5fe36b8-c065-44a9-a55b-582573671609
Co-authored-by: Amp <amp@ampcode.com>
2025-10-19 08:04:48 -07:00
Steve Yegge
790233f748 feat: Add 'bd stale' command to show and release orphaned executor claims
- Implements bd stale command to show issues with execution_state where executor is dead/stopped
- Adds --release flag to automatically release orphaned issues
- Adds --threshold flag to customize heartbeat staleness threshold (default: 300s/5min)
- Handles missing executor instances (LEFT JOIN) for cases where executor was deleted
- Adds QueryContext and BeginTx helper methods to SQLiteStorage for advanced queries
- Fixes ExternalRef comparison bug in import_shared.go (pointer vs string)
- Removes unused imports in import.go

Resolves vc-124
2025-10-18 17:14:21 -07:00
Steve Yegge
bb13f4634b Fix daemon crash recovery race conditions (bd-147)
Improvements based on oracle code review:
- Move socket cleanup AFTER lock acquisition (prevents unlinking live sockets)
- Add PID liveness check before removing stale socket
- Add stale lock detection with retry mechanism
- Tighten directory permissions to 0700 for security
- Improve socket readiness probing with shorter timeouts
- Make removeOldSocket() ignore ENOENT errors

Fixes race condition where socket could be removed during daemon startup window,
potentially orphaning a running daemon process.

Amp-Thread-ID: https://ampcode.com/threads/T-63542c60-b5b9-4a34-9f22-415d9d7e8223
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:59:06 -07:00
Steve Yegge
9e2ee1889f Add daemon health check endpoint (bd-146)
- Add OpHealth RPC operation to protocol
- Implement handleHealth() with DB ping and 1s timeout
- Returns status (healthy/degraded/unhealthy), uptime, cache metrics
- Update TryConnect() to use health check instead of ping
- Add 'bd daemon --health' CLI command with JSON output
- Track cache hits/misses for metrics
- Unhealthy daemon triggers automatic fallback to direct mode
- Health check completes in <2 seconds

Amp-Thread-ID: https://ampcode.com/threads/T-1a4889f3-77cf-433a-a704-e1c383929f48
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:41:06 -07:00
Steve Yegge
f987722f96 Code review fixes for cache eviction (bd-145)
Oracle-recommended improvements:

**Thread Safety:**
- Fix lastAccess race: updates now under Write lock
- Make Stop() idempotent with sync.Once
- Close stores synchronously (not in goroutine)

**Performance:**
- Replace O(n²) sort with sort.Slice (O(n log n))
- Enforce LRU immediately on insert (prevents FD spikes)

**Correctness:**
- Canonicalize cache key to repo root (not cwd)
  - Prevents duplicate connections for same repo
  - Multiple subdirs → single cache entry
- Validate env vars: TTL <= 0 falls back to default

**Tests (6 new edge cases):**
- Canonical key behavior across subdirectories
- Immediate LRU enforcement without periodic cleanup
- Invalid TTL handling
- Re-open after eviction
- Stop idempotency
- All tests pass with -race flag

This addresses potential data races, resource spikes, and duplicate
connections identified during code review.
2025-10-18 13:24:59 -07:00
Steve Yegge
259e994522 Add storage cache eviction policy to daemon (bd-145)
Implemented TTL-based and LRU cache eviction for daemon storage connections:

- Add StorageCacheEntry with lastAccess timestamp tracking
- Cleanup goroutine runs every 5 minutes to evict stale entries
- TTL-based eviction: remove entries idle >30min (configurable)
- LRU eviction: enforce max cache size (default: 50 repos)
- Configurable via BEADS_DAEMON_MAX_CACHE_SIZE and BEADS_DAEMON_CACHE_TTL
- Proper cleanup on server shutdown
- Update lastAccess on cache hits
- Comprehensive tests for eviction logic

Fixes memory leaks and file descriptor exhaustion for multi-repo users.

Amp-Thread-ID: https://ampcode.com/threads/T-1148d8b3-b8a8-45fc-af9c-b5be14c4834d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:17:07 -07:00
Steve Yegge
491cb82489 Add ready test for handling closed/blocked filter combinations
- Add test case for verifying --closed and --blocked filters don't apply to ready command
- Update issues.jsonl with new test-related issues

Amp-Thread-ID: https://ampcode.com/threads/T-1148d8b3-b8a8-45fc-af9c-b5be14c4834d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 13:12:14 -07:00
Steve Yegge
d2e34b863b Add --show-all-paths flag to bd dep tree
Implements bd-9: Allow users to view all paths through diamond dependencies
without deduplication. Useful for debugging complex dependency structures.

Changes:
- Added --show-all-paths flag to bd dep tree command
- Updated GetDependencyTree interface to accept showAllPaths parameter
- Modified deduplication logic to be conditional on flag
- Updated tests to pass new parameter

Amp-Thread-ID: https://ampcode.com/threads/T-43807dd5-8732-49ad-a839-cdb5dae70c35
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 11:52:57 -07:00
Steve Yegge
6811d1cf42 Fix multiple issues and renumber database
- bd-170: Implement hybrid sorting for ready work (recent 48h first, then oldest)
- bd-87: Use safer null-byte placeholders in ID remapping
- bd-92: Make auto-flush debounce configurable via BEADS_FLUSH_DEBOUNCE
- bd-171: Fix nil pointer dereference in renumber command
- Delete spurious test issues (bd-7, bd-130-134)
- Renumber database from 171 down to 144 issues
2025-10-18 09:58:35 -07:00
Steve Yegge
9d78a66dc3 new issues
Amp-Thread-ID: https://ampcode.com/threads/T-b66ca677-3810-44b1-bd13-4c887053f474
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 09:40:20 -07:00
Steve Yegge
fb9b5864af feat: Add bd repos multi-repo commands and fix bd ready for in_progress issues
- Add 'bd repos' command for multi-repository management (bd-123)
  - bd repos list: show all cached repositories
  - bd repos ready: aggregate ready work across repos
  - bd repos stats: combined statistics across repos
  - bd repos clear-cache: clear repository cache
  - Requires global daemon (bd daemon --global)

- Fix bd ready to show in_progress issues (bd-165)
  - bd ready now shows both 'open' and 'in_progress' issues with no blockers
  - Allows epics/tasks ready to close to appear in ready work
  - Critical P0 bug fix for workflow

- Apply code review improvements to repos implementation
  - Use strongly typed RPC responses (remove interface{})
  - Fix clear-cache lock handling (close connections outside lock)
  - Add error collection for per-repo failures
  - Add context timeouts (1-2s) to prevent hangs
  - Add lock strategy comments

- Update documentation (README.md, AGENTS.md)
- Add comprehensive tests for both features

Amp-Thread-ID: https://ampcode.com/threads/T-1de989a1-1890-492c-9847-a34144259e0f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-18 00:37:27 -07:00
Steve Yegge
0dac4b9003 Fix global daemon implementation and improve security
- bd-159: Global daemon now runs in routing mode without opening DB
- bd-158: Set socket permissions to 0600 for security
- bd-160: Reject --auto-commit/--auto-push with --global
- bd-157: Verified stale socket cleanup (already working)
- bd-56: Closed as won't-do (cycle prevention is better)
- bd-73: Multi-repo support complete
2025-10-17 23:17:22 -07:00
Steve Yegge
ee94d817ed feat: Add batch deletion support (bd-127)
- Add DeleteIssues() method in sqlite.go for atomic batch deletion
- Support multiple issue IDs as arguments or from file
- Add --from-file flag to read IDs from file (supports comments)
- Add --dry-run mode for safe preview without deleting
- Add --cascade flag for recursive deletion of dependents
- Add --force flag to orphan dependents instead of failing
- Pre-collect connected issues before deletion for text reference updates
- Add orphan deduplication to prevent duplicate IDs
- Add rows.Err() checks in all row iteration loops
- Full transaction safety - all deletions succeed or none do
- Comprehensive statistics tracking (deleted, dependencies, labels, events)
- Update README and CHANGELOG with batch deletion docs

Fixed critical code review issues:
- Dry-run mode now properly uses dryRun parameter instead of deleting data
- Text references are pre-collected before deletion so they update correctly
- Added orphan deduplication and error checks
- Updated defer rollback pattern per Go best practices
2025-10-17 21:13:23 -07:00
Steve Yegge
b40de9bc41 Implement daemon RPC with per-request context routing (bd-115)
- Added per-request storage routing in daemon server
  - Server now supports Cwd field in requests for database discovery
  - Tree-walking to find .beads/*.db from any working directory
  - Storage caching for performance across requests

- Created Python daemon client (bd_daemon_client.py)
  - RPC over Unix socket communication
  - Implements full BdClientBase interface
  - Auto-discovery of daemon socket from working directory

- Refactored bd_client.py with abstract interface
  - BdClientBase abstract class for common interface
  - BdCliClient for CLI-based operations (renamed from BdClient)
  - create_bd_client() factory with daemon/CLI fallback
  - Backwards-compatible BdClient alias

Next: Update MCP server to use daemon client when available
2025-10-17 16:28:29 -07:00
Steve Yegge
14c744861c Add epic closure management commands (fixes #62)
- Add 'bd epic status' to show epic completion with child progress
- Add 'bd epic close-eligible' to bulk-close completed epics
- Add GetEpicsEligibleForClosure() storage method
- Update 'bd stats' to show count of epics ready to close
- Add EpicStatus type for tracking epic/child relationships
- Support --eligible-only, --dry-run, and --json flags
- Fix golangci-lint config version requirement

Addresses GitHub issue #62 - epics now have visibility and
management tools for closure when all children are complete.

Amp-Thread-ID: https://ampcode.com/threads/T-e8ac3f48-f0cf-4858-8e8f-aace2481c30d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-17 13:50:20 -07:00
Juan Vargas
e6a69401c9 Add SQLite native time format to connection string (#61)
Use _time_format=sqlite parameter in modernc.org/sqlite connection
string to ensure DATETIME columns use SQLite's native time format
(format 7 with timezone) instead of Go's default String() format.

This improves compatibility with SQLite's date/time functions and
ensures consistent time representation across the database.
2025-10-17 11:35:43 -07:00
Steve Yegge
15b60b4ad0 Phase 4: Atomic operations and stress testing (bd-114, bd-110)
Completes daemon architecture implementation:

Features:
- Batch/transaction API (OpBatch) for multi-step atomic operations
- Request timeout and cancellation support (30s default, configurable)
- Comprehensive stress tests (4-10 concurrent agents, 800-1000 ops)
- Performance benchmarks (daemon 2x faster than direct mode)

Results:
- Zero ID collisions across 1000+ concurrent creates
- All acceptance criteria validated for bd-110
- Create: 2.4ms (daemon) vs 4.7ms (direct)
- Update/List: similar 2x improvement

Tests Added:
- TestStressConcurrentAgents (8 agents, 800 creates)
- TestStressBatchOperations (4 agents, 400 batch ops)
- TestStressMixedOperations (6 agents, mixed read/write)
- TestStressNoUniqueConstraintViolations (10 agents, 1000 creates)
- BenchmarkDaemonCreate/Update/List/Latency
- Fixed flaky TestConcurrentRequests (shared client issue)

Files:
- internal/rpc/protocol.go - Added OpBatch, BatchArgs, BatchResponse
- internal/rpc/server.go - Implemented handleBatch with stop-on-failure
- internal/rpc/client.go - Added SetTimeout and Batch methods
- internal/rpc/stress_test.go - All stress tests
- internal/rpc/bench_test.go - Performance benchmarks
- DAEMON_STRESS_TEST.md - Complete documentation

Closes bd-114, bd-110

Amp-Thread-ID: https://ampcode.com/threads/T-1c07c140-0420-49fe-add1-b0b83b1bdff5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 23:46:12 -07:00
Steve Yegge
5c0fac6e17 feat: Phase 1 RPC protocol infrastructure for daemon architecture (bd-111)
Implemented Unix socket RPC foundation to enable daemon-based concurrent access:

New files:
- internal/rpc/protocol.go: Request/Response types with 13 operations
- internal/rpc/server.go: Unix socket server with storage adapter
- internal/rpc/client.go: Client with auto-detection and typed methods
- internal/rpc/rpc_test.go: Integration tests

Features:
- JSON-based protocol over Unix sockets
- Adapter pattern for context/actor propagation to storage API
- Ping/health checks for daemon detection
- All core operations: create, update, close, list, show, ready, stats, deps, labels
- Graceful socket cleanup and signal handling
- Concurrent request support

Tests: 49.3% coverage, all passing

Related issues:
- bd-110: Daemon architecture epic
- bd-111: Phase 1 (completed)
- bd-112: Phase 2 (client auto-detection)
- bd-113: Phase 3 (daemon command)
- bd-114: Phase 4 (atomic operations)

Amp-Thread-ID: https://ampcode.com/threads/T-796c62e6-93b6-41c7-9cb5-8acc4a35ba9a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 22:49:19 -07:00
Steve Yegge
b87ef26b22 Fix renumber counter: Force reset to actual max ID
The counter wasn't being properly reset after renumbering because
SyncAllCounters uses MAX(old, new) which kept higher values from
deleted issues.

Solution: Add ResetCounter() method to delete the counter entry,
then SyncAllCounters recreates it from the actual max ID in database.

Now after renumbering 108 issues to bd-1..bd-108, the counter is
correctly set to 108 and next issue will be bd-109.
2025-10-16 21:51:35 -07:00
Steve Yegge
872f203c57 Add RPC infrastructure and updated database
- RPC Phase 1: Protocol, server, client implementation
- Updated renumber.go with proper text reference updates (3-phase approach)
- Clean database exported: 344 issues (bd-1 to bd-344)
- Added DAEMON_DESIGN.md documentation
- Updated go.mod/go.sum for RPC dependencies

Amp-Thread-ID: https://ampcode.com/threads/T-456af77c-8b7f-4004-9027-c37b95e10ea5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 20:36:23 -07:00
Steve Yegge
f32d90af4e Implement bd delete command with comprehensive cleanup 2025-10-16 19:18:23 -07:00
Steve Yegge
fc29beae6d Fix bd-421: Add deduplication to prevent importing duplicate issues
- Added deduplicateIncomingIssues() to consolidate content-identical issues
- DetectCollisions now deduplicates within incoming batch before processing
- Keeps issue with smallest ID when duplicates found
- Added comprehensive test suite in collision_dedup_test.go
- Export clean JSONL with bd-421 fix applied

Amp-Thread-ID: https://ampcode.com/threads/T-c17dd8bf-c298-4a80-baa5-55fa7c7bb9a3
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 18:08:58 -07:00
Steve Yegge
2abccb7a88 Implement bd restore command and flip ready work sort order
- Add bd restore command to view full history of compacted issues from git
- Command temporarily checks out historical commit, reads JSONL, displays original content
- Read-only operation, no database or git state modification
- Flip ready work sort to created_at ASC (older issues first within priority tier)
- Prevents issue treadmill effect, surfaces old P1s for triage
- Update README.md and AGENTS.md with restore documentation

Closes bd-407, bd-383
2025-10-16 18:06:53 -07:00
Steve Yegge
65f59e6b01 Add compacted_at_commit field and git commit capture during compaction
- Add compacted_at_commit field to Issue type (bd-405)
- Add database schema and migration for new field
- Create GetCurrentCommitHash() helper function
- Update ApplyCompaction to store git commit hash (bd-395)
- Update compaction calls to capture current commit
- Update tests to verify commit hash storage
- All tests passing

Amp-Thread-ID: https://ampcode.com/threads/T-5518cccb-7fc9-4dcd-ba5a-e22cd10e45d7
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 17:43:38 -07:00
Steve Yegge
363cd3b4e6 Add rename-prefix command (bd-420)
- Implement bd rename-prefix command with --dry-run and --json flags
- Add prefix validation (max 8 chars, lowercase, starts with letter)
- Update all issue IDs and text references atomically per issue
- Update dependencies, labels, events, and counters
- Fix counter merge to use MAX() to prevent ID collisions
- Update snapshot tables for FK integrity
- Add comprehensive tests for validation and rename workflow
- Document in README.md and AGENTS.md

Known limitation: Each issue updates in its own transaction.
A failure mid-way could leave mixed state. Acceptable for
intended use case (infrequent operation on small DBs).

Amp-Thread-ID: https://ampcode.com/threads/T-7e77b779-bd88-44f2-9f0b-a9f2ccd54d38
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 17:05:27 -07:00
Steve Yegge
c3e3326bba Fix critical bugs: bd-169, bd-28, bd-393
- bd-169: Add -q/--quiet flag to bd init command
- bd-28: Improve error handling in RemoveDependency
  - Now checks RowsAffected and returns error if dependency doesn't exist
  - New removeDependencyIfExists() helper for collision remapping
- bd-393: CRITICAL - Fix auto-import skipping collisions
  - Auto-import was LOSING work from other workers
  - Now automatically remaps collisions to new IDs
  - Calls RemapCollisions() instead of skipping

All tests pass.

Amp-Thread-ID: https://ampcode.com/threads/T-cba86837-28db-47ce-94eb-67fade82376a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 15:00:54 -07:00
Steve Yegge
4fe648d140 Fix bd-437: Use addDependencyUnchecked during collision resolution
When remapping dependencies during collision resolution, skip semantic
validation (like parent-child direction checks) since we're just updating
IDs on existing dependencies that were already validated.

Fixes parent-child validation error during import --resolve-collisions.
2025-10-16 13:42:47 -07:00
Steve Yegge
21bd7809b5 Add cycle detection performance benchmarks (bd-311)
- Created comprehensive benchmark suite for cycle detection
- Tested linear chains, tree structures, and dense graphs
- Results: 3-4ms overhead per AddDependency is acceptable
- Documented findings in test file and DESIGN.md
- Closed bd-311 and epic bd-307
2025-10-16 13:32:44 -07:00
Steve Yegge
1e32041fe6 Implement cycle detection and prevention improvements
- Add diagnostic warnings when cycles detected after dep add (bd-309)
- Add semantic validation for parent-child dependency direction (bd-308)
- Document cycle handling behavior in code, README, and DESIGN (bd-310)

Changes:
- cmd/bd/dep.go: Add DetectCycles() call and warning after dep add
- internal/storage/sqlite/dependencies.go: Add parent-child direction validation and comprehensive cycle prevention comments
- internal/storage/sqlite/dependencies_test.go: Add TestParentChildValidation
- README.md: Add dependency types and cycle prevention section with examples
- DESIGN.md: Add detailed cycle prevention design rationale and trade-offs
2025-10-16 13:18:07 -07:00
Steve Yegge
331a435418 Fix collision resolver: sync ID counters before remapping
The collision resolver was failing when remapping issues to new IDs because
the issue_counters table was out of sync with actual database IDs. Test issues
(bd-200 through bd-312) were created without updating the counter, causing
UNIQUE constraint violations when --resolve-collisions tried to use those IDs.

Added SyncAllCounters() call before remapping to ensure counters reflect all
existing issues in the database. This prevents ID collisions during import.

Fixes the core functionality needed for multi-device git-based sync (bd-271).

Amp-Thread-ID: https://ampcode.com/threads/T-a2a94e1b-b220-41b0-bf7d-f2640a44292b
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 11:59:34 -07:00
Steve Yegge
b7f9e74574 feat: Implement cross-type cycle prevention in dependencies
- Remove type filter from cycle detection to check ALL dependency types
- Extract maxDependencyDepth=100 constant shared across AddDependency and DetectCycles
- Move cycle check before INSERT to avoid unnecessary write on failure
- Add comprehensive tests: self-dependency, related cycles, cross-type cycles
- Verify idx_dependencies_issue index exists for performance

Fixes bd-312. Prevents cross-type cycles (e.g., A blocks B, B parent-child A)
that previously hid work from ready list. Addresses oracle feedback for
proper implementation.
2025-10-16 10:59:25 -07:00
Steve Yegge
e99de1b5e3 Add EventCompacted to event system (bd-262)
- Add EventCompacted event type constant
- Add compaction fields to Issue struct (CompactionLevel, CompactedAt, OriginalSize)
- Update ApplyCompaction to record compaction events with JSON metadata
- Update bd show to display compaction status with emoji indicators
- Update GetIssue query to load compaction fields
- All tests passing

Amp-Thread-ID: https://ampcode.com/threads/T-3f7946c6-8f5e-4a81-9527-1217041c7b39
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 01:00:27 -07:00
Steve Yegge
da5493bac0 Remove snapshot/restore functionality from compaction
Snapshots defeated the entire purpose of compaction - if we're keeping
the original content, we're not actually saving any space. Compaction
is about graceful memory decay for agentic databases, not reversible
compression.

Removed:
- CreateSnapshot/GetSnapshots/RestoreFromSnapshot from storage
- --restore flag and functionality from bd compact command
- All snapshot-related tests
- Snapshot struct and related code

The database is ephemeral and meant to decay over time. Compaction
actually reduces database size now.

Closes bd-260 (won't fix - conceptually wrong)
Closes bd-261 (already done in bd-259)
2025-10-16 00:26:42 -07:00
Steve Yegge
35a4cba829 Add bd compact CLI command (bd-259)
Implements the compact command with all required features:
- --dry-run: Preview compaction with size estimates
- --all: Process all eligible candidates
- --id: Compact specific issue
- --force: Bypass eligibility checks (requires --id)
- --stats: Show compaction statistics
- --tier: Select compaction tier (1 or 2)
- --workers: Configure parallel workers
- --batch-size: Configure batch processing
- Progress bar with visual feedback
- JSON output support
- Proper exit codes and error handling
- Summary reporting (count, bytes saved, reduction %, time)

Includes additional test coverage for compaction and snapshot operations.

Amp-Thread-ID: https://ampcode.com/threads/T-ffcaf749-f79c-4b03-91dd-42136b2744b1
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 00:13:14 -07:00
Steve Yegge
e2c42c35c4 Add label and title filtering to bd list (closes #45, bd-269)
- Add --label/-l flag to filter issues by labels (AND logic)
- Add --title flag to filter issues by title substring
- Add TitleSearch field to IssueFilter type
- Implement label and title filtering in SearchIssues
- Perfect for worktree-specific issue management

Examples:
  bd list --label worktree,feature-x
  bd list --title "authentication"
  bd list --label worktree --title "bug"
2025-10-15 23:51:57 -07:00
Joshua Shanks
96f05a79cb Fix linter errors in CreateIssues after upstream merge
- Fix unchecked error return in CreateIssues ROLLBACK (errcheck)
- Reduce cyclomatic complexity of CreateIssues from 28 to ~10 (gocyclo)
  - Extract validateBatchIssues helper for validation phase
  - Extract generateBatchIDs helper for ID generation
  - Extract bulkInsertIssues helper for issue insertion
  - Extract bulkRecordEvents helper for event recording
  - Extract bulkMarkDirty helper for dirty marking

All refactored helpers maintain the same functionality and transaction safety.
Follows the same pattern used in main.go complexity reduction.
2025-10-15 23:38:47 -07:00
Joshua Shanks
b1e8ef556e Refactor main.go to reduce cyclomatic complexity
Breaks down large functions into smaller, focused helpers to pass gocyclo linter:

Auto-import refactoring:
- Extract parseJSONLIssues() to handle JSONL parsing
- Extract handleCollisions() to detect and report conflicts
- Extract importIssueData() to coordinate issue/dep/label imports
- Extract updateExistingIssue() and createNewIssue() for clarity
- Extract importDependencies() and importLabels() for modularity

Flush refactoring:
- Extract recordFlushFailure() and recordFlushSuccess() for state management
- Extract readExistingJSONL() to isolate file reading logic
- Extract fetchDirtyIssuesFromDB() to separate DB access
- Extract writeIssuesToJSONL() to handle atomic writes

Command improvements:
- Extract executeLabelCommand() to eliminate duplication in label.go
- Extract addLabelsToIssue() helper for label management
- Replace deprecated strings.Title with manual capitalization

Configuration:
- Add gocyclo exception for test files in .golangci.yml

All tests passing, no functionality changes.
2025-10-15 23:38:47 -07:00
Joshua Shanks
cf4f11cff7 Fix error handling consistency in auto-import and fallback paths (#47)
* Fix error handling consistency in auto-import and fallback paths

- Add error checking/warnings for auto-import CRUD operations (UpdateIssue, CreateIssue, AddDependency)
- Add error checking/warnings for auto-import label operations (AddLabel, RemoveLabel)
- Add warning when import hash storage fails (prevents unnecessary re-imports)
- Add proper error handling for UserHomeDir with fallback to current directory

These changes make auto-import error handling consistent with manual operations
and prevent silent failures that could confuse users or cause data inconsistencies.

* Remove invalid version property from golangci-lint config

* Fix linter errors: errcheck, unused, goconst, and misspell

- Fix unchecked error returns in ROLLBACK statements
- Fix unchecked type assertion for status field
- Extract LIMIT SQL constant to reduce duplication
- Fix spelling: cancelled -> canceled
- Remove unused ensureCounterInitialized function
- Remove unused parameter in parallel test goroutine
2025-10-15 23:34:33 -07:00
Steve Yegge
0da81371b4 Implement Tier 1 compaction logic (bd-257)
- Add Compactor with CompactTier1 and CompactTier1Batch methods
- Single issue and batch compaction with 5 concurrent workers
- Dry-run mode for testing without API calls
- Smart size checking: keeps original if summary is longer
- Improved Haiku prompts to emphasize compression
- Add ApplyCompaction method for setting compaction metadata
- Comprehensive tests including API integration tests
- All tests passing
2025-10-15 23:31:43 -07:00
Steve Yegge
5f6aac5fb1 Implement snapshot creation and restoration for compaction (bd-256)
- Add compaction_snapshots table to schema with proper indexes
- Implement CreateSnapshot, RestoreFromSnapshot, GetSnapshots functions
- Use UTC timestamps throughout
- RestoreFromSnapshot uses transactions with optimistic concurrency control
- Add validation for levels and issue_id matching
- Prevent race conditions with compaction_level guard
- Create bd-268 to explore lightweight SQL alternatives

Amp-Thread-ID: https://ampcode.com/threads/T-3bdd0d6b-9212-4e4e-b22d-f658949df7a9
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 23:20:21 -07:00
Steve Yegge
3e3f46d6d2 Add Claude Haiku client for issue compaction (bd-255)
- Implement HaikuClient with Tier 1 and Tier 2 summarization prompts
- Add exponential backoff retry logic for transient errors (network, 429, 5xx)
- API key precedence: env var overrides explicit key parameter
- Comprehensive tests with UTF-8 support and edge case coverage
- Retry logic only retries transient errors, stops on permanent failures
- Clarified retry semantics: maxRetries=3 means 4 total attempts (1 initial + 3 retries)
2025-10-15 23:09:33 -07:00
Steve Yegge
1c5a4a9c70 Add compaction schema and candidate identification
- Added compaction columns to issues table (compaction_level, compacted_at, original_size)
- Created issue_snapshots table for snapshot storage before compaction
- Added compaction configuration with opt-in flag (compaction_enabled=false by default)
- Implemented GetTier1Candidates and GetTier2Candidates queries
- Added CheckEligibility validation function
- Comprehensive tests for all compaction queries
- Idempotent migrations for existing databases

Closes bd-252, bd-253, bd-254

Amp-Thread-ID: https://ampcode.com/threads/T-c4d7acd1-c161-4b80-9d80-a0691e8fa87b
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 22:26:11 -07:00
Steve Yegge
347100319e Document CreateIssues API (bd-243)
- Add comprehensive godoc to CreateIssues with usage examples
- Add batch operations section to EXTENDING.md with performance comparison
- Add performance feature to README.md
- Include when to use CreateIssue vs CreateIssues guidance
- Document counter sync requirement after explicit IDs

Amp-Thread-ID: https://ampcode.com/threads/T-59fc78c3-a7f2-4c33-b074-2fa840c97c87
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 20:24:07 -07:00
Steve Yegge
5ab7ff0f84 Add comprehensive unit tests for CreateIssues (bd-241)
- Added 12 table-driven test cases covering all validation scenarios
- Added 2 rollback tests (validation error + DB conflict)
- Added nil item detection to prevent panic
- Fixed nil pointer panic in CreateIssues
- Tests verify ID uniqueness, timestamps, closed_at invariant
- All tests pass

Amp-Thread-ID: https://ampcode.com/threads/T-61c584cd-d873-4a1a-bfa6-d739b630b3e5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 19:33:53 -07:00
Steve Yegge
e7d4a9c822 Implement CreateIssues batch API (bd-240, bd-244)
- Add CreateIssues method to Storage interface
- Implement SQLiteStorage.CreateIssues with atomic ID range reservation
- Single transaction for N issues (5-10x speedup expected)
- Set timestamps before validation to match CreateIssue behavior
- All tests passing
2025-10-15 19:13:27 -07:00