Commit Graph

393 Commits

Author SHA1 Message Date
Steve Yegge
d7f4189e3e feat: Add 'bd count' command for counting and grouping issues
Implements a new 'bd count' command that provides efficient issue counting
with filtering and grouping capabilities.

Features:
- Basic count: Returns total count of issues matching filters
- All filtering options from 'bd list' (status, priority, type, assignee, labels, dates, etc.)
- Grouping via --by-* flags: status, priority, type, assignee, label
- JSON output support for both simple and grouped counts
- Both daemon and direct mode support

Implementation:
- Added OpCount operation and CountArgs to RPC protocol
- Added Count() method to RPC client
- Implemented handleCount() server-side handler with optimized bulk label fetching
- Created cmd/bd/count.go with full CLI implementation

Performance optimization:
- Pre-fetches all labels in a single query when using --by-label to avoid N+1 queries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 19:21:05 -05:00
Marco Del Pin
f724b612d2 fix: Deduplicate database paths when symlinks present (#354)
Fixes a bug where FindAllDatabases() would return the same database
multiple times when symlinks were present in the directory hierarchy.

The issue occurred because the function walked up the directory tree
without resolving symlinks or checking if a database had already been
discovered. This led to confusing warnings about "multiple databases"
even when only one physical database existed.

Changes:
- Added symlink resolution using filepath.EvalSymlinks()
- Implemented deduplication using a seen map with canonical paths
- Added comprehensive tests for symlink scenarios

This ensures that each unique database is reported exactly once,
regardless of how many symlinks point to it in the directory tree.

Resolves duplicate database warnings when symlinks are present.
2025-11-21 12:22:33 -08:00
Steve Yegge
3a0446c431 Fix test compilation errors: add context.Background() to sqlite.New() calls
Updated all test files to pass context.Background() as the first parameter
to sqlite.New() calls to match the updated function signature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 14:57:18 -05:00
Steve Yegge
1168f661d1 fix: Add context.Background() to sqlite.New() calls in test files
Multiple test files were still using the old sqlite.New(path) signature
instead of the new sqlite.New(ctx, path) signature. This was causing
compilation failures in the test suite.

Fixed files:
- internal/importer/importer_test.go
- internal/importer/external_ref_test.go
- internal/importer/timestamp_test.go
- internal/rpc/limits_test.go
- internal/rpc/list_filters_test.go
- internal/rpc/rpc_test.go
- internal/rpc/status_test.go
- internal/syncbranch/syncbranch_test.go
2025-11-21 14:48:41 -05:00
Steve Yegge
82902432f5 test: Tag 16 slow integration tests with build tags
Identified and tagged obviously-slow integration tests with
`//go:build integration` to exclude them from default test runs.

This is step 1 of fixing test performance. The real fix is in
bd-1rh: refactoring tests to use shared DB setup instead of
creating 279 separate databases.

Tagged files:
- cmd/bd: 8 files (CLI tests, git ops, performance benchmarks)
- internal: 8 files (integration tests, E2E tests)

Issues:
- bd-1rh: Main issue tracking test performance
- bd-c49: Audit all tests and create grouping plan (next step)
- bd-y6d: POC refactor of create_test.go

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 14:40:22 -05:00
Steve Yegge
e816e91ecb Complete bd-ar2 P2 tasks: metadata, resurrection, and testing improvements
This commit addresses the remaining P2 tasks from bd-ar2 code review follow-up:

## Completed Tasks

### bd-ar2.4: Improve parent chain resurrection
- Modified `tryResurrectParentWithConn()` to recursively resurrect ancestor chain
- When resurrecting bd-root.1.2, now also resurrects bd-root.1 if missing
- Handles deeply nested hierarchies where intermediate parents are deleted
- All resurrection tests pass including new edge cases

### bd-ar2.5: Add error handling guidance
- Documented metadata update failure strategy in `updateExportMetadata()`
- Explained trade-off: warnings vs errors (safe, prevents data loss)
- Added user-facing message: "Next export may require running 'bd import' first"
- Clarifies that worst case is requiring import before next export

### bd-ar2.6: Document transaction boundaries
- Added comprehensive documentation for atomicity trade-offs
- Explained crash scenarios and recovery (bd import)
- Documented decision to defer defensive checks (Option 4) until needed
- No code changes - current approach is acceptable for now

### bd-ar2.12: Add metadata key validation
- Added keySuffix validation in `updateExportMetadata()` and `hasJSONLChanged()`
- Prevents ':' separator in keySuffix to avoid malformed metadata keys
- Documented metadata key format in function comments
- Single-repo: "last_import_hash", multi-repo: "last_import_hash:<repo_key>"

### bd-ar2.7: Add edge case tests for GetNextChildID resurrection
- TestGetNextChildID_ResurrectParent_NotInJSONL: parent not in history
- TestGetNextChildID_ResurrectParent_NoJSONL: missing JSONL file
- TestGetNextChildID_ResurrectParent_MalformedJSONL: invalid JSON lines
- TestGetNextChildID_ResurrectParentChain: deeply nested missing parents
- All tests pass, resurrection is robust against edge cases

## Files Changed
- cmd/bd/daemon_sync.go: Metadata validation, error handling docs
- cmd/bd/integrity.go: Added strings import, keySuffix validation
- internal/storage/sqlite/hash_ids.go: Improved resurrection comments
- internal/storage/sqlite/resurrection.go: Recursive ancestor resurrection
- internal/storage/sqlite/child_id_test.go: Added 4 new edge case tests

## Testing
All export, sync, metadata, and resurrection tests pass.
Edge cases properly handled: missing JSONL, malformed JSON, deep nesting.

## Remaining Tasks
- bd-ar2.8 (P3): Additional export metadata edge case tests (deferred)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 11:40:37 -05:00
Steve Yegge
cd8cb8b86a Fix bd-ar2 code review issues: metadata tracking and multi-repo support
This commit addresses critical code review findings from bd-dvd and bd-ymj fixes:

## Completed Tasks

### bd-ar2.1: Extract duplicated metadata update code
- Created `updateExportMetadata()` helper function
- Eliminated 22-line duplication between createExportFunc and createSyncFunc
- Single source of truth for metadata updates

### bd-ar2.2: Add multi-repo support to export metadata updates
- Added per-repo metadata key tracking with keySuffix parameter
- Both export and sync functions now update metadata for all repos

### bd-ar2.3: Fix tests to use actual daemon functions
- TestExportUpdatesMetadata now calls updateExportMetadata() directly
- Added TestUpdateExportMetadataMultiRepo() for multi-repo testing
- Fixed export_mtime_test.go tests to call updateExportMetadata()

### bd-ar2.9: Fix variable shadowing in GetNextChildID
- Changed `err` to `resurrectErr` to avoid shadowing
- Improves code clarity and passes linter checks

### bd-ar2.10: Fix hasJSONLChanged to support per-repo keys
- Updated hasJSONLChanged() to accept keySuffix parameter
- Reads metadata with correct per-repo keys
- All callers updated (validatePreExport, daemon import, sync command)

### bd-ar2.11: Use stable repo identifiers instead of paths
- Added getRepoKeyForPath() helper function
- Uses stable identifiers like ".", "../frontend" instead of absolute paths
- Metadata keys now portable across machines and clones
- Prevents orphaned metadata when repos are moved

## Files Changed
- cmd/bd/daemon_sync.go: Helper functions, metadata updates
- cmd/bd/integrity.go: hasJSONLChanged() with keySuffix support
- cmd/bd/sync.go: Updated to use getRepoKeyForPath()
- cmd/bd/*_test.go: Tests updated for new signatures
- internal/storage/sqlite/hash_ids.go: Fixed variable shadowing

## Testing
All export, sync, and integrity tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 11:28:25 -05:00
Steve Yegge
4c5f99c5bd Fix bd-dvd and bd-ymj: Parent resurrection and export metadata
Bug 1 (bd-dvd): GetNextChildID now attempts parent resurrection from JSONL
before failing. Added TryResurrectParent call to match CreateIssue behavior.

Bug 2 (bd-ymj): Export now updates last_import_hash metadata to prevent
'JSONL content has changed' errors on subsequent exports.

Files changed:
- internal/storage/sqlite/hash_ids.go: Add resurrection attempt
- cmd/bd/daemon_sync.go: Add metadata updates after export
- Tests added for both fixes
- Fixed pre-existing bug in integrity_content_test.go

Follow-up work tracked in epic bd-ar2 (9 issues for improvements).

Fixes GH #334
2025-11-21 10:29:30 -05:00
Steve Yegge
57253f93a3 Context propagation with graceful cancellation (bd-rtp, bd-yb8, bd-2o2)
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.

Key changes:

1. Signal-aware contexts (bd-rtp):
   - Added rootCtx/rootCancel in main.go using signal.NotifyContext()
   - Set up in PersistentPreRun, cancelled in PersistentPostRun
   - Daemon uses same pattern in runDaemonLoop()
   - Handles SIGINT/SIGTERM for graceful shutdown

2. Context propagation (bd-yb8):
   - All commands now use rootCtx instead of context.Background()
   - sqlite.New() receives context for cancellable operations
   - Database operations respect context cancellation
   - Storage layer propagates context through all queries

3. Cancellation tests (bd-2o2):
   - Added import_cancellation_test.go with comprehensive tests
   - Added export cancellation test in export_test.go
   - Tests verify database integrity after cancellation
   - All cancellation tests passing

Fixes applied during review:
   - Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
   - Fixed test context contamination (reset rootCtx in test cleanup)
   - Fixed export tests missing context setup

Impact:
   - Pressing Ctrl+C during import/export now cancels gracefully
   - No database corruption or hanging transactions
   - Clean shutdown of all operations

Tested:
   - go build ./cmd/bd ✓
   - go test ./cmd/bd -run TestImportCancellation ✓
   - go test ./cmd/bd -run TestExportCommand ✓
   - Manual Ctrl+C testing verified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 21:57:23 -05:00
Steve Yegge
9088988edd Improve staleness check error handling and optimization (bd-n4td, bd-o4qy, bd-c4rq)
This commit implements three related improvements to database staleness checking:

**bd-n4td (P2): Add warning when staleness check errors**
- Added stderr warnings when CheckStaleness fails in ensureDatabaseFresh
- Users now see "Warning: could not check database staleness: <error>"
- Provides visibility into staleness check failures while allowing operations to proceed

**bd-o4qy (P2): Improve CheckStaleness error handling**
- Updated CheckStaleness to distinguish between expected and abnormal conditions:
  * Returns (false, nil) for expected "no data yet" scenarios (missing metadata, missing JSONL)
  * Returns (false, err) for abnormal errors (glob failures, permission errors)
- Updated RPC server (2 locations) to log staleness errors but allow requests to proceed
- Prevents blocking operations due to transient staleness check issues
- Added comprehensive function documentation

**bd-c4rq (P3): Refactor staleness check to avoid function call overhead**
- Moved daemon check from inside ensureDatabaseFresh to all 8 call sites
- Avoids unnecessary function call when running in daemon mode
- Updated: list.go, info.go, status.go, show.go, stale.go, duplicates.go, ready.go, validate.go
- Extracted staleness functions to new staleness.go for better organization

**Code review fixes:**
- Removed dead code in CheckStaleness (unreachable jsonlPath == "" check)
- Removed unused ensureDatabaseFreshQuiet function

**Files changed:**
- New: cmd/bd/staleness.go (extracted staleness checking functions)
- Modified: 8 command files (added daemon check before staleness calls)
- Modified: internal/autoimport/autoimport.go (improved error handling)
- Modified: internal/rpc/server_export_import_auto.go (handle errors gracefully)
2025-11-20 20:45:39 -05:00
Steve Yegge
fb1d12fe36 bd cleanup: Remove 493 closed issues
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 20:39:29 -05:00
Steve Yegge
d5239eeef9 Refactor: Extract duplicated validation logic to internal/validation
Extracted repeated priority and ID validation patterns from CLI commands
into reusable functions in internal/validation/bead.go.

Changes:
- Added ValidatePriority(): Combines parsing and error handling
- Added ValidateIDFormat(): Validates ID format and extracts prefix
- Added ValidatePrefix(): Validates prefix matching with database config
- Updated create.go and show.go to use new validation functions
- Simplified force flag logic to always call ValidatePrefix()
- Added comprehensive tests for all validation functions
- Added TODO comment for daemon mode validation enhancement

Results:
- Reduced code duplication by ~20 lines
- Centralized validation logic for easier maintenance
- Consistent error messages across all commands
- All tests passing

Fixes bd-g5p7

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 20:39:23 -05:00
Steve Yegge
968d9e2ea1 Optimize bd list: replace N+1 label queries with bulk fetch
Problem:
In direct mode, bd list was making a separate GetLabels() call for
each issue when displaying labels. With 538 issues, this resulted in
538 separate database queries.

While investigating the reported 5+ second slowness, discovered this
N+1 query issue that would impact performance with many issues.

Solution:
1. Added GetLabelsForIssues(issueIDs []string) to Storage interface
2. Implemented bulk fetch in SQLite (already existed, now exposed)
3. Implemented bulk fetch in MemoryStorage
4. Updated list.go to fetch all labels in single query

Changes:
- internal/storage/storage.go: Add GetLabelsForIssues to interface
- internal/storage/memory/memory.go: Implement GetLabelsForIssues
- cmd/bd/list.go: Use bulk fetching in all output modes

Impact:
Eliminates N queries for labels, replacing with 1 bulk query.
This optimization applies to direct mode only (daemon mode already
uses bulk operations via RPC).

Note: The reported 5s slowness was actually caused by daemon auto-start
timeout. Use --no-daemon flag or run 'bd migrate --update-repo-id' to
resolve the legacy database issue causing daemon startup failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 20:27:09 -05:00
Steve Yegge
614ba8ab20 Add cache invalidation for blocked_issues_cache
Ensures cache stays synchronized with dependency and status changes
by calling invalidateBlockedCache() at all mutation points (bd-5qim).

Cache invalidation points:
- AddDependency: when type is 'blocks' or 'parent-child'
- RemoveDependency: when removed dep was 'blocks' or 'parent-child'
- UpdateIssue: when status field changes
- CloseIssue: always (closed issues don't block)

The invalidation strategy is full cache rebuild on any change,
which is fast enough (<1ms for 10K issues) and keeps the logic simple.

Only 'blocks' and 'parent-child' dependency types affect blocking,
so 'relates-to' and other types skip invalidation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:29:30 -05:00
Steve Yegge
ed23f8f4fe Optimize GetReadyWork to use blocked_issues_cache
Replaces expensive recursive CTE query with simple cache lookup,
achieving 96% performance improvement on 10K databases (bd-5qim).

Performance results:
- Before: ~752ms (recursive CTE on every call)
- After: ~29ms (cache lookup + filters)
- Target: <50ms ✓

The query now uses a simple NOT EXISTS check against the
blocked_issues_cache table instead of computing the full
blocked issue tree on every call.

Cache is maintained by invalidateBlockedCache() called on
dependency and status changes (added in next commit).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:29:30 -05:00
Steve Yegge
62c1f42d9f Add blocked_issues_cache table for GetReadyWork optimization
Introduces a materialized cache table to store blocked issue IDs,
replacing the expensive recursive CTE computation that was causing
~752ms query times on 10K databases (bd-5qim).

The cache is maintained via invalidation on dependency and status
changes, reducing GetReadyWork from O(n²) recursive traversal to
O(1) cache lookup.

Technical details:
- New blocked_issues_cache table with single issue_id column
- ON DELETE CASCADE ensures automatic cleanup
- Migration populates cache using existing recursive CTE logic
- rebuildBlockedCache() fully rebuilds cache on invalidation
- execer interface allows both *sql.DB and *sql.Tx usage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:29:30 -05:00
Steve Yegge
0470105fee Merge branch 'main' of https://github.com/steveyegge/beads
# Conflicts:
#	.beads/issues.jsonl
2025-11-20 19:19:28 -05:00
Steve Yegge
3b2cac4d8f Centralize error handling patterns in storage layer (bd-bwk2)
Created internal/storage/sqlite/errors.go with:
- Sentinel errors: ErrNotFound, ErrInvalidID, ErrConflict, ErrCycle
- wrapDBError helpers that auto-convert sql.ErrNoRows to ErrNotFound
- Type-safe error checking with errors.Is() compatibility

Updated error handling across storage layer:
- dirty.go: Added context to error returns, converted sql.ErrNoRows checks
- util.go: Updated withTx to use wrapDBError
- batch_ops.go: Added context wrapping to batch operations
- dependencies.go: Wrapped errors from markIssuesDirtyTx calls
- ids.go: Added error wrapping for ID validation

Also restored sqlite.go that was accidentally deleted in previous commit.

All tests pass. Provides consistent error wrapping with operation context
for better debugging.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:17:57 -05:00
Steve Yegge
06b6f864b8 Merge remote-tracking branch 'origin/main'
Amp-Thread-ID: https://ampcode.com/threads/T-c8d369a3-32f0-42a0-96d1-fd589e89bd6b
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 19:12:06 -05:00
Steve Yegge
bbfedb060a Refactor: extract duplicated validation and flag logic (bd-g5p7)
- Created internal/validation package for centralized validation logic
- Created cmd/bd/flags.go for shared flag registration
- Updated create and update commands to use shared logic
- Added support for 'P1' style priority to update command
- Added tests for validation logic

Amp-Thread-ID: https://ampcode.com/threads/T-c8d369a3-32f0-42a0-96d1-fd589e89bd6b
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 19:11:27 -05:00
Steve Yegge
1736f9f12d bd sync: 2025-11-20 18:58:07 2025-11-20 19:10:37 -05:00
Steve Yegge
631c9236e7 Resolve merge conflict in beads.jsonl
Amp-Thread-ID: https://ampcode.com/threads/T-fc47ce9d-88a4-4bcd-b9cb-79327d98dee7
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 19:08:23 -05:00
Steve Yegge
8e05847d31 Fix TestRoutingIntegration and improve DetectUserRole robustness
Amp-Thread-ID: https://ampcode.com/threads/T-fc47ce9d-88a4-4bcd-b9cb-79327d98dee7
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 19:04:42 -05:00
Steve Yegge
6a25f5eaa6 fix: resolve remaining hyphenated ID issues in memory store and doctor 2025-11-20 19:01:30 -05:00
Steve Yegge
7b865eb541 Remove legacy issues.jsonl and update defaults to beads.jsonl
Amp-Thread-ID: https://ampcode.com/threads/T-ae11b392-24b5-4060-b431-606dd81c1763
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 19:00:36 -05:00
Steve Yegge
345766badc Fix FK constraint failures in AddComment and ApplyCompaction (bd-5arw)
Amp-Thread-ID: https://ampcode.com/threads/T-4358e6e4-28ea-4ed7-ba3f-3da39072e169
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 18:55:10 -05:00
Steve Yegge
7665ff0763 Merge branch 'fix-issue-325' into main
Amp-Thread-ID: https://ampcode.com/threads/T-aa765a68-5cc4-465b-a2f6-aa008933c11e
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 14:20:45 -05:00
Steve Yegge
09666b4219 Fix FOREIGN KEY constraint failed when operating on non-existent issues
Fixes #325

- Fix CloseIssue to check rows affected before inserting event
- Fix RemoveLabel to check rows affected before inserting event
- Fix UpdateIssueID to check rows affected before inserting event
- Prevent orphan events and confusing error messages

Amp-Thread-ID: https://ampcode.com/threads/T-aa765a68-5cc4-465b-a2f6-aa008933c11e
Co-authored-by: Amp <amp@ampcode.com>
2025-11-20 12:35:26 -05:00
Steve Yegge
20ffe60316 Merge origin/main 2025-11-20 12:27:47 -05:00
Steve Yegge
78a893d17d Merge pull request #337 from cpdata/fix-resolve-partial-id
Fix: Restore JSON unmarshaling in ResolveID for fixing failed bd show/update/close operations
2025-11-20 09:15:08 -08:00
Ryan
fb65163692 fix: address critical resource leaks and error handling issues (#327)
* fix: address critical resource leaks and error handling issues

Fixes 5 critical and high-priority issues identified in codebase analysis:

1. bd-vavh: Fix row iterator resource leak in recursive dependency queries
   - Move defer rows.Close() to execute on all code paths
   - Previously leaked connections on scan errors
   - Location: internal/storage/sqlite/sqlite.go:1121-1145

2. bd-qhws: Configure database connection pool limits for daemon mode
   - Set MaxOpenConns to runtime.NumCPU() + 1 for file-based databases
   - Prevents connection exhaustion under concurrent RPC load
   - Only affects daemon mode (long-running server)
   - Location: internal/storage/sqlite/sqlite.go:108-125

3. bd-jo38: Add WaitGroup tracking to FileWatcher goroutines
   - Track goroutines with sync.WaitGroup for graceful shutdown
   - Wait for goroutines to finish before cleanup in Close()
   - Prevents race condition on debouncer access during shutdown
   - Location: cmd/bd/daemon_watcher.go (Start, startPolling, Close)

4. bd-2d5r: Fix silent error handling in RPC response writing
   - writeResponse now returns errors instead of ignoring them
   - Prevents sending partial JSON and client hangs
   - Closes connection on marshal/write errors
   - Location: internal/rpc/server_lifecycle_conn.go:227-246

5. bd-zqmb: Fix goroutine leak in daemon restart
   - Add 10-second timeout to daemon Wait() goroutine
   - Kill process if it doesn't fork within timeout
   - Prevents goroutine accumulation on restart failures
   - Location: cmd/bd/daemons.go:250-268

All changes follow Go best practices and maintain backward compatibility.

* Add feature request for .beads/README.md generation during init

Created bd-m7ge to automatically generate a promotional/documentation
README in the .beads directory when running 'bd init'. This will help
advertise Beads in open source repositories and provide quick reference
documentation for developers using AI coding agents.

The README will include:
- Brief explanation of Beads (AI-native issue tracking)
- Link to steveyegge/beads repository
- Quick reference of essential commands
- Compelling messaging to encourage adoption
2025-11-20 08:13:06 -08:00
Charles P. Cross
4432af0aa4 fix: improve ResolvePartialID / ResolveID handling for bd show (issue #336)
- Add fast path for exact ID matches in ResolvePartialID
- Properly unmarshal ResolveID RPC responses used by `bd show`

Summary:
--------
Fixes regression introduced after v0.23.1 where `bd show <issue_id>`,
`bd update <issue_id>`, and `bd close <issue_id>` commands failed with
"operation failed: issue not found" errors even when the issue existed
in the database.

Root Cause:
-----------
The daemon's ResolveID RPC handler (handleResolveID in
internal/rpc/server_issues_epics.go) returns a JSON-encoded string
containing the resolved issue ID. For example, when resolving "ao-izl",
the RPC response contains the JSON string: "ao-izl" (with quotes).

After v0.23.1, the show/update/close commands in cmd/bd/show.go were
changed to use string(resp.Data) to extract the resolved ID, which
treats the raw bytes as a string without decoding the JSON. This caused
the resolved ID to literally be "ao-izl" (including the quotes),
which then failed to match the actual issue ID ao-izl (without quotes)
when passed to subsequent Show/Update/Close RPC calls.

In v0.23.1 (commit 77dcf55), the code correctly unmarshaled the JSON:
  var resolvedID string
  if err := json.Unmarshal(resp.Data, &resolvedID); err != nil {
      // handle error
  }

This unmarshal step was removed in later commits, causing the regression.

Fix:
----
Restored the JSON unmarshaling step in three places in cmd/bd/show.go:
1. showCmd - line 37-42
2. updateCmd - line 400-405
3. closeCmd - line 723-728

Each now properly decodes the JSON string response before using the
resolved ID in subsequent RPC calls.

Testing:
--------
- Verified `bd show <issue-id-with-prefix>` works correctly in both daemon and direct modes
- Tested with newly created issues to ensure the fix works for all IDs
- All Go tests pass (go test ./... -short)
- Confirmed behavior matches v0.23.1 working binary

The fix ensures that issue ID resolution works correctly regardless of
whether the ID prefix matches the configured issue_prefix.
2025-11-19 05:54:28 -05:00
Codex Agent
4cd26c8e5a Fix Windows concurrent issue creation 2025-11-17 10:41:05 -07:00
Codex Agent
bf9b2c83fb Annotate gosec-safe file accesses 2025-11-17 10:12:46 -07:00
Codex Agent
7b63b5a30b Fix CI regressions and stabilize tests 2025-11-17 10:06:35 -07:00
Steve Yegge
934ae04fa0 Fix gh-316: Prefer exact ID matches over prefix matches
The ID disambiguation logic treated 'offlinebrew-3d0' as ambiguous when
child IDs like 'offlinebrew-3d0.1' existed. Now the system:

1. Checks for exact full ID matches first (issue.ID == input)
2. Checks for exact hash matches (handling cross-prefix scenarios)
3. Only falls back to substring matching if no exact match is found

Added test cases verifying:
- 'offlinebrew-3d0' matches exactly, not ambiguously with children
- '3d0' without prefix still resolves to exact match

Amp-Thread-ID: https://ampcode.com/threads/T-5358ea57-e9ea-49e9-aedf-7044ebf8b52a
Co-authored-by: Amp <amp@ampcode.com>
2025-11-15 14:07:38 -08:00
Steve Yegge
b9919fe031 Fix: --parent flag now creates parent-child dependency
When using 'bd create --parent <id>', the system now automatically
creates a parent-child dependency linking the child to the parent.
This fixes epic status reporting which was showing zero children.

Fixes #318 (bd-jijf)

Changes:
- cmd/bd/create.go: Add parent-child dependency after issue creation
- internal/rpc/server_issues_epics.go: Add same fix for daemon mode

Tested:
- Created epic with children, verified epic status shows correct count
- Verified closing all children makes epic eligible for closure
- All tests pass

Amp-Thread-ID: https://ampcode.com/threads/T-f1a1aee1-03bd-4e62-a63c-c1d339f8300b
Co-authored-by: Amp <amp@ampcode.com>
2025-11-15 13:20:02 -08:00
Ryan
690c73fc31 Performance Improvements (#319)
* feat: add performance testing framework foundation

Implements foundation for comprehensive performance testing and user
diagnostics for beads databases at 10K-20K scale.

Components added:
- Fixture generator (internal/testutil/fixtures/) for realistic test data
  * LargeSQLite/XLargeSQLite: 10K/20K issues with epic hierarchies
  * LargeFromJSONL/XLargeFromJSONL: test JSONL import path
  * Realistic cross-linked dependencies, labels, assignees
  * Reproducible with seeded RNG

- User diagnostics (bd doctor --perf) for field performance data
  * Collects platform info (OS, arch, Go/SQLite versions)
  * Measures key operation timings (ready, list, show, search)
  * Generates CPU profiles for bug reports
  * Clean separation in cmd/bd/doctor/perf.go

Test data characteristics:
- 10% epics, 30% features, 60% tasks
- 4-level hierarchies (Epic → Feature → Task → Subtask)
- 20% cross-epic blocking dependencies
- Realistic status/priority/label distributions

Supports bd-l954 (Performance Testing Framework epic)
Closes bd-6ed8, bd-q59i

* perf: optimize GetReadyWork with compound index (20x speedup)

Add compound index on dependencies(depends_on_id, type, issue_id) to
eliminate performance bottleneck in GetReadyWork recursive CTE query.

Performance improvements (10K issue database):
- GetReadyWork: 752ms → 36.6ms (20.5x faster)
- Target: <50ms ✓ ACHIEVED
- 20K database: ~1500ms → 79.4ms (19x faster)

Benchmark infrastructure enhancements:
- Add dataset caching in /tmp/beads-bench-cache/ to avoid regenerating
  10K-20K issues on every benchmark run (first run: ~2min, subsequent: <5s)
- Add progress logging during fixture generation (shows 10%, 20%... completion)
- Add database size logging (17.5 MB for 10K, 35.1 MB for 20K)
- Document rationale for only benchmarking large datasets (>10K issues)
- Add CPU/trace profiling with --profile flag for performance debugging

Schema changes:
- internal/storage/sqlite/schema.go: Add idx_dependencies_depends_on_type_issue

New files:
- internal/storage/sqlite/bench_helpers_test.go: Reusable benchmark setup with caching
- internal/storage/sqlite/sqlite_bench_test.go: Comprehensive benchmarks for critical operations
- Makefile: Convenient benchmark execution (make bench-quick, make bench)

Related:
- Resolves bd-5qim (optimize GetReadyWork performance)
- Builds on bd-6ed8 (fixture generator), bd-q59i (bd doctor --perf)

* perf: add WASM compilation cache to eliminate cold-start overhead

Configure wazero compilation cache for ncruces/go-sqlite3 to avoid
~220ms JIT compilation on every process start.

Cache configuration:
- Location: ~/.cache/beads/wasm/ (platform-specific via os.UserCacheDir)
- Automatic version management: wazero keys entries by its version
- Fallback: in-memory cache if directory creation fails
- No cleanup needed: old versions are harmless (~5-10MB each)

Performance impact:
- First run: ~220ms (populate cache)
- Subsequent runs: ~20ms (load from cache)
- Savings: ~200ms per cold start

Cache invalidation:
- Automatic when wazero version changes (upgrades use new cache dir)
- Manual cleanup: rm -rf ~/.cache/beads/wasm/ (safe to delete anytime)

This complements daemon mode:
- Daemon mode: eliminates startup cost by keeping process alive
- WASM cache: reduces startup cost for one-off commands or daemon restarts

Changes:
- internal/storage/sqlite/sqlite.go: Add init() with cache setup

* refactor: improve maintainability of performance testing code

Extract common patterns and eliminate duplication across benchmarks, fixture generation, and performance diagnostics. Replace magic numbers with explicit configuration to improve readability and make it easier to tune test parameters.

* docs: clarify profiling behavior and add missing documentation

Add explanatory comments for profiling setup to clarify why --profile
forces direct mode (captures actual database operations instead of RPC
overhead) and document the stopCPUProfile function's role in flushing
profile data to disk. Also fix gosec G104 linter warning by explicitly
ignoring Close() error during cleanup.

* fix: prevent bench-quick from running indefinitely

Added //go:build bench tags and skipped timeout-prone benchmarks to
prevent make bench-quick from running for hours.

Changes:
- Add //go:build bench tag to cycle_bench_test.go and compact_bench_test.go
- Skip Dense graph benchmarks (documented to timeout >120s)
- Fix compact benchmark prefix: bd- → bd (validation expects prefix without trailing dash)

Before: make bench-quick ran for 3.5+ hours (12,699s) before manual interrupt
After: make bench-quick completes in ~25 seconds

The Dense graph benchmarks are known to timeout and represent rare edge
cases that don't need optimization for typical workflows.
2025-11-15 12:46:13 -08:00
Steve Yegge
944ed1033d Fix bd-yvlc: in-memory database deadlock in migrations
- Force single connection for all in-memory databases (including file::memory:)
- Close rows before executing statements in external_ref migration
- Prevents connection pool deadlock with MaxOpenConns(1)
- Fixes test failures in syncbranch_test.go
2025-11-15 12:42:02 -08:00
David Laing
57b6ea606b fix: add external_ref support to daemon mode RPC (fixes #303) (#304)
Add external_ref field to CreateArgs and UpdateArgs RPC protocol
structs to enable linking issues to external systems (GitHub, Jira,
Shortcut, etc.) when using daemon mode.

Changes:
- Add ExternalRef field to rpc.CreateArgs and rpc.UpdateArgs
- Update bd create/update commands to pass external_ref via RPC
- Update daemon handlers to process external_ref field
- Add integration tests for create and update operations

The --external-ref flag now works correctly in both daemon and direct modes.

Fixes https://github.com/steveyegge/beads/issues/303

Generated with [Claude Code](https://claude.com/claude-code)
via [Happy](https://happy.engineering)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2025-11-13 12:01:27 -08:00
Ryan
f7e80dd80c feat: add context optimization features for AI agents (#297)
* feat: add bd prime and setup commands for AI agent integration

This commit consolidates context optimization features for AI agents:

## New Commands

**bd prime** - AI-optimized workflow context injection
- Outputs ~1-2k tokens of workflow context
- Context-aware: adapts to MCP vs CLI mode
- MCP mode: minimal reminders (~500 tokens)
- CLI mode: full command reference (~1-2k tokens)
- Warns against TodoWrite tool and markdown TODOs
- Designed for SessionStart/PreCompact hooks

**bd setup claude** - Claude Code integration installer
- Installs hooks via JSON configuration (not file scripts)
- Supports --project for project-only installation
- Supports --check to verify installation
- Supports --remove to uninstall hooks
- Idempotent (safe to run multiple times)
- Merges with existing settings

**bd setup cursor** - Cursor IDE integration installer
- Creates .cursor/rules/beads.mdc with workflow rules
- Simplified implementation (just overwrites file)

## bd doctor Enhancements

- New: CheckClaude() verifies Claude Code integration
- Detects plugin, MCP server, and hooks installation
- Provides actionable fix suggestions
- Extracted legacy pattern detection to doctor/legacy.go
- Detects JSONL-only mode and warns about legacy issues.jsonl

## Core Improvements

- FindBeadsDir() utility for cross-platform .beads/ discovery
- Works in JSONL-only mode (no database required)
- Sorted noDbCommands alphabetically (one per line for easy diffs)

## Testing

- Unit tests for setup command hook manipulation
- Tests for idempotency, adding/removing hooks
- All tests passing

## Documentation

- cmd/bd/doctor/claude.md - Documents why beads doesn't use Claude Skills
- commands/prime.md - Slash command for bd prime
- Fixed G304 gosec warnings with nosec comments

## Token Efficiency

The bd prime approach reduces AI context usage dramatically:
- MCP mode: ~500 tokens (vs ~10.5k for full MCP tool scan)
- CLI mode: ~1-2k tokens
- 80-99% reduction in standing context overhead

* fix: resolve linting errors in setup utils and remove obsolete test

- Add error check for tmpFile.Close() in setup/utils.go to fix golangci-lint G104
- Remove TestCheckMultipleJSONLFiles test that referenced deleted checkMultipleJSONLFiles function

Fixes golangci-lint errcheck violations introduced in the bd prime/setup feature.
2025-11-12 10:48:36 -08:00
Steve Yegge
8be792a460 Fix external_ref migration failure on old databases
The schema initialization was trying to create an index on the external_ref
column before the migration that adds the column runs. This caused 'no such
column: external_ref' errors when opening very old databases (pre-0.17.5).

Solution: Move the index creation into the migration that adds the column.

Fixes #284

Amp-Thread-ID: https://ampcode.com/threads/T-2744d5a7-168f-4ef6-bcab-926db846de20
Co-authored-by: Amp <amp@ampcode.com>
2025-11-10 10:50:39 -08:00
Steve Yegge
83472aca3d bd sync: 2025-11-09 14:53:59 2025-11-09 14:53:59 -08:00
Steve Yegge
98f2e85618 Fix version test error message assertion 2025-11-08 22:57:49 -08:00
Steve Yegge
734579b1a2 Remove version field from metadata.json
- Removes noisy version mismatch warnings on every bd upgrade
- Version field in metadata.json was redundant with daemon version checking via RPC
- Daemon version mismatches still detected via HealthResponse
- Removes checkVersionMismatch() function and related test file
- Updates .beads/.gitignore to properly ignore merge artifacts

Amp-Thread-ID: https://ampcode.com/threads/T-7ba8aff2-97a0-4d0c-9008-e858bdfadd61
Co-authored-by: Amp <amp@ampcode.com>
2025-11-08 18:16:39 -08:00
Steve Yegge
f027de93b6 Add schema compatibility probe to prevent silent migration failures (bd-ckvw)
- Implement comprehensive schema probe in sqlite.New() that verifies all
  expected tables and columns after migrations
- Add retry logic: if probe fails, retry migrations once
- Return clear fatal error with missing schema elements if probe still fails
- Enhance daemon version gating: refuse RPC if client has newer minor version
- Improve checkVersionMismatch messaging: verify schema before claiming upgrade
- Add schema compatibility check to bd doctor command
- Add comprehensive tests for schema probing

This prevents the silent migration failure bug where:
1. Migrations fail silently
2. Database queries fail with 'no such column' errors
3. Import logic misinterprets as 'not found' and tries INSERT
4. Results in cryptic UNIQUE constraint errors

Fixes #262

Amp-Thread-ID: https://ampcode.com/threads/T-0d7ae2c0-9f12-4f9b-85d1-1291488af150
Co-authored-by: Amp <amp@ampcode.com>
2025-11-08 15:40:19 -08:00
Steve Yegge
773aa736e4 Document external_ref in content hash behavior (bd-9f4a)
- Added comprehensive code comments in collision.go explaining external_ref inclusion
- Documented content hash behavior in HASH_ID_DESIGN.md with examples
- Enhanced test documentation in collision_test.go
- Closes bd-9f4a, bd-df11, bd-537e

Amp-Thread-ID: https://ampcode.com/threads/T-47525168-d51c-4f56-b598-18402e5ea389
Co-authored-by: Amp <amp@ampcode.com>
2025-11-08 02:22:15 -08:00
Steve Yegge
a140436db8 test: improve internal/daemon test coverage to 60%
Adds fast unit tests for previously uncovered functions in the daemon package:

- checkDaemonErrorFile: tests reading daemon error files
- StopDaemon: tests error handling for non-running daemons
- KillAllDaemons: tests empty lists and non-alive daemons
- FindDaemonByWorkspace: tests not found case
- discoverDaemon: tests missing socket scenario
- CleanupStaleSockets: tests edge cases (already removed, alive daemon)
- Registry: tests corrupted file handling and unregistering non-existent entries

Coverage improved from 22.5% to 60.0% with only fast tests (<1s runtime).
All new tests work in -short mode and don't start actual daemons.

Fixes bd-3f80d9e0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-08 01:20:03 -08:00
Steve Yegge
f6dbcd1a4f Add test safeguards to prevent production database pollution (bd-2c5a)
- Add failIfProductionDatabase() check in Go test helpers
- Add temp directory verification in RPC test setup
- Create conftest.py with pytest safety checks for Python tests
- Add BEADS_TEST_MODE env var to mark test execution
- Tests now fail fast if they detect production .beads/ usage

This prevents test issues from polluting the production database
like the incident on Nov 7, 2025 where 29+ test issues were created
in .beads/beads.db instead of isolated test databases.

Resolves: bd-2c5a
Amp-Thread-ID: https://ampcode.com/threads/T-635a8807-1120-4122-a0cb-4c21970362ce
Co-authored-by: Amp <amp@ampcode.com>
2025-11-07 21:58:22 -08:00
Steve Yegge
fe705befbe Add RPC diagnostics with BD_RPC_DEBUG env var
- Add BD_RPC_DEBUG=1 for lightweight timing logs to stderr
- Log socket path, socket exists check, dial timing, health check timing
- Improve daemon status message when lock not held
- Helps field triage of connection issues without verbose daemon logs
- Fixes bd-j7e2
2025-11-07 21:29:22 -08:00