- Add validation in ValidateWithCustomStatuses() requiring deleted_at for tombstones
- Add validation that non-tombstones cannot have deleted_at set
- Block direct status update to tombstone in validateStatusWithCustom()
- Users must use 'bd delete' instead of 'bd update --status=tombstone'
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: use os.Lstat for symlink-safe mtime and permission checks
On NixOS and other systems using symlinks heavily (e.g., home-manager),
os.Stat follows symlinks and returns the target's metadata. This causes:
1. False staleness detection when JSONL is symlinked - mtime of target
changes unpredictably when symlinks are recreated
2. os.Chmod failing or changing wrong file's permissions when target
is in read-only location (e.g., /nix/store)
3. os.Chtimes modifying target's times instead of the symlink itself
Changes:
- autoimport.go: Use Lstat for JSONL mtime in CheckStaleness()
- import.go: Use Lstat in TouchDatabaseFile() for JSONL mtime
- export.go: Skip chmod for symlinked files
- multirepo.go: Use Lstat for JSONL mtime cache
- multirepo_export.go: Use Lstat for mtime, skip chmod for symlinks
- doctor/fix/permissions.go: Skip permission fixes for symlinked paths
These changes are safe cross-platform:
- On systems without symlinks, Lstat behaves identically to Stat
- Symlink permission bits are ignored on Unix anyway
- The extra Lstat syscall overhead is negligible
Fixes symlink-related data loss on NixOS. See GitHub issue #379.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* test: add symlink behavior tests for NixOS compatibility
Add tests that verify symlink handling behavior:
- TestCheckStaleness_SymlinkedJSONL: verifies mtime detection uses
symlink's own mtime (os.Lstat), not target's mtime (os.Stat)
- TestPermissions_SkipsSymlinkedBeadsDir: verifies chmod is skipped
for symlinked .beads directories
- TestPermissions_SkipsSymlinkedDatabase: verifies chmod is skipped
for symlinked database files while still fixing .beads dir perms
Also adds devShell to flake.nix for local development with go, gopls,
golangci-lint, and sqlite tools.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Also updated CONFIG.md to clarify mass delete threshold requires >5 issues (bd-in6).
The string(rune('0'+i)) pattern produces incorrect characters when i >= 10.
Changed to strconv.Itoa(i) for reliable conversion.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
After adding close_reason column to issues table, two functions still
called GetCloseReason() to fetch from events table after already
scanning the column. Removed the redundant code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add migration 017_close_reason_column.go to create the column
- Update all INSERT statements to include close_reason
- Update all SELECT statements to include close_reason
- Update doctor.go to check for close_reason in schema validation
- Remove workaround code that batch-loaded close reasons from events table
- Fix migrations_test.go to include close_reason in test table schema
This fixes sync loops where close_reason values were silently dropped
because the DB lacked the column despite the struct having the field.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
SQLite WAL mode writes go to the -wal file, not the main database.
Without an explicit checkpoint before Close(), writes can be stranded
and lost between CLI invocations.
This was causing `bd migrate` to report success but not actually
persist the version update to the database.
Fixes#434🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add -u/--unassigned flag to bd ready command to show only issues
with no assignee. This supports the SCAVENGE protocol where polecats
query the 'Salvage Yard' for unassigned ready work.
Changes:
- Add NoAssignee field to WorkFilter struct
- Update SQLite GetReadyWork to filter by empty/null assignee
- Add --unassigned/-u flag to ready command
- Update RPC protocol and daemon handler
- Add test for NoAssignee filter functionality
Fixes: gt-3rp
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds the ability to filter ready work for issues with no assignee,
which is useful for the SCAVENGE protocol in Gas Town where polecats
need to query the "Salvage Yard" for unclaimed work.
Changes:
- Add Unassigned bool field to types.WorkFilter
- Add --unassigned/-u flag to bd ready command
- Update SQL query in GetReadyWork to filter for NULL/empty assignee
- Add Unassigned field to RPC ReadyArgs for daemon support
- Add tests for the new functionality
Closes: gt-3rp
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* bd sync: 2025-11-29 00:08:58
* fix(multirepo): handle out-of-order dependencies during JSONL import
Fixes#413. When importing issues from multi-repo JSONL files, if issue A
(line 1) has a dependency on issue B (line 5), the import would fail with
FK constraint error because B doesn't exist yet.
Solution:
- Disable FK checks at start of importJSONLFile()
- Re-enable FK checks before commit
- Run PRAGMA foreign_key_check to validate data integrity
- Fail with clear error if orphaned dependencies are detected
This allows out-of-order dependencies while still catching corrupted data.
---------
Co-authored-by: Shaun Cutts <shauncutts@factfiber.com>
Users can now define custom status states for multi-step pipelines using:
bd config set status.custom "awaiting_review,awaiting_testing,awaiting_docs"
Changes:
- Add Status.IsValidWithCustom() method for custom status validation
- Add Issue.ValidateWithCustomStatuses() method
- Add GetCustomStatuses() method to storage interface
- Update CreateIssue/UpdateIssue to support custom statuses
- Add comprehensive tests for custom status functionality
- Update config command help text with custom status documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add CloseReason field to Issue struct
- Add GetCloseReason and GetCloseReasonsForIssues queries
- Batch-load close reasons in scanIssues for efficiency
- Display close reason in bd show output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When auto-importing issues from JSONL, issues with different prefixes
(e.g., gt-1 vs gastown-) would fail validation and cause an infinite
loop of failed migrations.
The fix adds SkipPrefixValidation option to CreateIssuesWithFullOptions
which propagates through EnsureIDs to skip prefix validation for issues
that already have IDs during import. This allows importing issues with
any prefix while still validating new issues created interactively.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add safe type assertions in applyUpdatesToIssue (bd-4gs)
- Add --sort and --reverse flags to bd search (bd-4f6)
- Add test cases for SearchIssues priority range, date range, IDs (bd-ew5)
- Handle errors from GetLabelsForIssues in search.go (bd-lce)
- Standardize error wrapping to fmt.Errorf pattern (bd-7kl)
- Extract shared scanIssueRow helper function (bd-ajf)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive documentation for the blocked_issues_cache optimization
that improved GetReadyWork performance from 752ms to 29ms (25x speedup).
Documentation locations:
- blocked_cache.go: Detailed package comment covering architecture,
invalidation strategy, transaction safety, edge cases, and future
optimizations
- ready.go: Enhanced comment at query site explaining the optimization
and maintenance triggers
- ARCHITECTURE.md: New section with diagrams, blocking semantics,
performance characteristics, and testing instructions
Closes bd-1w6i
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add exponential backoff retry for BEGIN IMMEDIATE transactions to handle
concurrent write load without spurious failures.
Changes:
- Add IsBusyError() helper to detect database locked errors
- Add beginImmediateWithRetry() with exponential backoff (10ms, 20ms, 40ms, 80ms, 160ms)
- Update CreateIssue and CreateIssuesInBatch to use retry logic
- Add comprehensive tests for error detection and retry behavior
- Handles context cancellation between retry attempts
- Fails fast on non-busy errors
This eliminates spurious SQLITE_BUSY failures under normal concurrent usage
while maintaining proper error handling for other failure modes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Standardized error handling across the SQLite storage layer by
consistently using wrapDBError() helper functions that were already
defined in errors.go.
Changes:
- config.go: Applied wrapDBError to all config/metadata functions
- queries.go: Fixed bare 'return err' in CreateIssue, UpdateIssue, DeleteIssues
- store.go: Changed %v to %w for proper error chain preservation
- errors_test.go: Added comprehensive test coverage for error wrapping
All error paths now:
- Wrap errors with operation context using %w
- Convert sql.ErrNoRows to ErrNotFound consistently
- Preserve error chains for unwrapping and type checking
This improves debugging by maintaining operation context throughout
the error chain and enables type-safe error checking with sentinel
errors.
All tests passing ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Creates migration to detect orphaned child issues and logs them for user
action. Orphaned children are issues with hierarchical IDs (e.g., "parent.child")
where the parent issue no longer exists in the database.
The migration:
- Queries for issues with IDs like '%.%' where parent doesn't exist
- Logs detected orphans with suggested actions (delete, convert, or restore)
- Does NOT automatically delete or convert orphans
- Is idempotent and safe to run multiple times
Test coverage:
- Detects orphaned child issues correctly
- Handles clean databases with no orphans
- Verifies idempotency
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
CHANGES:
1. Merge logic (internal/merge/merge.go):
- Added mergeStatus() enforcing closed ALWAYS wins over open
- Fixed closed_at handling: only set when status='closed'
- Changed deletion handling: deletion ALWAYS wins over modification
2. Deletion tracking (cmd/bd/snapshot_manager.go):
- Updated ComputeAcceptedDeletions to accept all merge deletions
- Removed "unchanged locally" check (deletion wins regardless)
3. FK constraint helper (internal/storage/sqlite/util.go):
- Added IsForeignKeyConstraintError() for bd-koab
- Detects FK violations for graceful import handling
TESTS UPDATED:
- TestMergeStatus: comprehensive status merge tests
- TestIsForeignKeyConstraintError: FK constraint detection
- bd-pq5k test: validates no invalid state (status=open with closed_at)
- Deletion tests: reflect new deletion-wins behavior
- All tests pass ✓
This ensures issues never get stuck in invalid states and prevents
the insane situation where issues never die!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
**Problem**: Export operations called GetLabels() and GetIssueComments()
in a loop for each issue, creating N+1 query pattern. For 100 issues
this created 201 queries instead of 3-5.
**Solution**:
- Added GetCommentsForIssues() batch method to storage interface
- Implemented batch method in SQLite and memory storage backends
- Updated handleExport() and triggerExport() to use batch queries
- Added comprehensive tests for batch operations
**Impact**: Query count reduced from ~201 to ~3-5 for 100 issues.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed compilation errors in benchmark test files where `ctx` was
declared twice, preventing benchmarks from running.
Changes:
- internal/storage/sqlite/bench_helpers_test.go: Remove duplicate ctx declaration
- internal/storage/sqlite/compact_bench_test.go: Remove duplicate ctx declaration
This allows `go test -tags=bench` to compile and run successfully.
Related to bd-5qim verification.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The test was failing on Windows CI because of insufficient time resolution
between creating an issue and adding a comment. Both operations could
complete within the same time unit, causing identical timestamps.
Added a 2ms sleep between operations to ensure updated_at is strictly
after the original timestamp, even on systems with lower time resolution.
Fixes: bd-pi7u
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Identified and tagged obviously-slow integration tests with
`//go:build integration` to exclude them from default test runs.
This is step 1 of fixing test performance. The real fix is in
bd-1rh: refactoring tests to use shared DB setup instead of
creating 279 separate databases.
Tagged files:
- cmd/bd: 8 files (CLI tests, git ops, performance benchmarks)
- internal: 8 files (integration tests, E2E tests)
Issues:
- bd-1rh: Main issue tracking test performance
- bd-c49: Audit all tests and create grouping plan (next step)
- bd-y6d: POC refactor of create_test.go
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses critical code review findings from bd-dvd and bd-ymj fixes:
## Completed Tasks
### bd-ar2.1: Extract duplicated metadata update code
- Created `updateExportMetadata()` helper function
- Eliminated 22-line duplication between createExportFunc and createSyncFunc
- Single source of truth for metadata updates
### bd-ar2.2: Add multi-repo support to export metadata updates
- Added per-repo metadata key tracking with keySuffix parameter
- Both export and sync functions now update metadata for all repos
### bd-ar2.3: Fix tests to use actual daemon functions
- TestExportUpdatesMetadata now calls updateExportMetadata() directly
- Added TestUpdateExportMetadataMultiRepo() for multi-repo testing
- Fixed export_mtime_test.go tests to call updateExportMetadata()
### bd-ar2.9: Fix variable shadowing in GetNextChildID
- Changed `err` to `resurrectErr` to avoid shadowing
- Improves code clarity and passes linter checks
### bd-ar2.10: Fix hasJSONLChanged to support per-repo keys
- Updated hasJSONLChanged() to accept keySuffix parameter
- Reads metadata with correct per-repo keys
- All callers updated (validatePreExport, daemon import, sync command)
### bd-ar2.11: Use stable repo identifiers instead of paths
- Added getRepoKeyForPath() helper function
- Uses stable identifiers like ".", "../frontend" instead of absolute paths
- Metadata keys now portable across machines and clones
- Prevents orphaned metadata when repos are moved
## Files Changed
- cmd/bd/daemon_sync.go: Helper functions, metadata updates
- cmd/bd/integrity.go: hasJSONLChanged() with keySuffix support
- cmd/bd/sync.go: Updated to use getRepoKeyForPath()
- cmd/bd/*_test.go: Tests updated for new signatures
- internal/storage/sqlite/hash_ids.go: Fixed variable shadowing
## Testing
All export, sync, and integrity tests pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Bug 1 (bd-dvd): GetNextChildID now attempts parent resurrection from JSONL
before failing. Added TryResurrectParent call to match CreateIssue behavior.
Bug 2 (bd-ymj): Export now updates last_import_hash metadata to prevent
'JSONL content has changed' errors on subsequent exports.
Files changed:
- internal/storage/sqlite/hash_ids.go: Add resurrection attempt
- cmd/bd/daemon_sync.go: Add metadata updates after export
- Tests added for both fixes
- Fixed pre-existing bug in integrity_content_test.go
Follow-up work tracked in epic bd-ar2 (9 issues for improvements).
Fixes GH #334
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.
Key changes:
1. Signal-aware contexts (bd-rtp):
- Added rootCtx/rootCancel in main.go using signal.NotifyContext()
- Set up in PersistentPreRun, cancelled in PersistentPostRun
- Daemon uses same pattern in runDaemonLoop()
- Handles SIGINT/SIGTERM for graceful shutdown
2. Context propagation (bd-yb8):
- All commands now use rootCtx instead of context.Background()
- sqlite.New() receives context for cancellable operations
- Database operations respect context cancellation
- Storage layer propagates context through all queries
3. Cancellation tests (bd-2o2):
- Added import_cancellation_test.go with comprehensive tests
- Added export cancellation test in export_test.go
- Tests verify database integrity after cancellation
- All cancellation tests passing
Fixes applied during review:
- Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
- Fixed test context contamination (reset rootCtx in test cleanup)
- Fixed export tests missing context setup
Impact:
- Pressing Ctrl+C during import/export now cancels gracefully
- No database corruption or hanging transactions
- Clean shutdown of all operations
Tested:
- go build ./cmd/bd ✓
- go test ./cmd/bd -run TestImportCancellation ✓
- go test ./cmd/bd -run TestExportCommand ✓
- Manual Ctrl+C testing verified
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Ensures cache stays synchronized with dependency and status changes
by calling invalidateBlockedCache() at all mutation points (bd-5qim).
Cache invalidation points:
- AddDependency: when type is 'blocks' or 'parent-child'
- RemoveDependency: when removed dep was 'blocks' or 'parent-child'
- UpdateIssue: when status field changes
- CloseIssue: always (closed issues don't block)
The invalidation strategy is full cache rebuild on any change,
which is fast enough (<1ms for 10K issues) and keeps the logic simple.
Only 'blocks' and 'parent-child' dependency types affect blocking,
so 'relates-to' and other types skip invalidation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replaces expensive recursive CTE query with simple cache lookup,
achieving 96% performance improvement on 10K databases (bd-5qim).
Performance results:
- Before: ~752ms (recursive CTE on every call)
- After: ~29ms (cache lookup + filters)
- Target: <50ms ✓
The query now uses a simple NOT EXISTS check against the
blocked_issues_cache table instead of computing the full
blocked issue tree on every call.
Cache is maintained by invalidateBlockedCache() called on
dependency and status changes (added in next commit).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Introduces a materialized cache table to store blocked issue IDs,
replacing the expensive recursive CTE computation that was causing
~752ms query times on 10K databases (bd-5qim).
The cache is maintained via invalidation on dependency and status
changes, reducing GetReadyWork from O(n²) recursive traversal to
O(1) cache lookup.
Technical details:
- New blocked_issues_cache table with single issue_id column
- ON DELETE CASCADE ensures automatic cleanup
- Migration populates cache using existing recursive CTE logic
- rebuildBlockedCache() fully rebuilds cache on invalidation
- execer interface allows both *sql.DB and *sql.Tx usage
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Created internal/storage/sqlite/errors.go with:
- Sentinel errors: ErrNotFound, ErrInvalidID, ErrConflict, ErrCycle
- wrapDBError helpers that auto-convert sql.ErrNoRows to ErrNotFound
- Type-safe error checking with errors.Is() compatibility
Updated error handling across storage layer:
- dirty.go: Added context to error returns, converted sql.ErrNoRows checks
- util.go: Updated withTx to use wrapDBError
- batch_ops.go: Added context wrapping to batch operations
- dependencies.go: Wrapped errors from markIssuesDirtyTx calls
- ids.go: Added error wrapping for ID validation
Also restored sqlite.go that was accidentally deleted in previous commit.
All tests pass. Provides consistent error wrapping with operation context
for better debugging.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: address critical resource leaks and error handling issues
Fixes 5 critical and high-priority issues identified in codebase analysis:
1. bd-vavh: Fix row iterator resource leak in recursive dependency queries
- Move defer rows.Close() to execute on all code paths
- Previously leaked connections on scan errors
- Location: internal/storage/sqlite/sqlite.go:1121-1145
2. bd-qhws: Configure database connection pool limits for daemon mode
- Set MaxOpenConns to runtime.NumCPU() + 1 for file-based databases
- Prevents connection exhaustion under concurrent RPC load
- Only affects daemon mode (long-running server)
- Location: internal/storage/sqlite/sqlite.go:108-125
3. bd-jo38: Add WaitGroup tracking to FileWatcher goroutines
- Track goroutines with sync.WaitGroup for graceful shutdown
- Wait for goroutines to finish before cleanup in Close()
- Prevents race condition on debouncer access during shutdown
- Location: cmd/bd/daemon_watcher.go (Start, startPolling, Close)
4. bd-2d5r: Fix silent error handling in RPC response writing
- writeResponse now returns errors instead of ignoring them
- Prevents sending partial JSON and client hangs
- Closes connection on marshal/write errors
- Location: internal/rpc/server_lifecycle_conn.go:227-246
5. bd-zqmb: Fix goroutine leak in daemon restart
- Add 10-second timeout to daemon Wait() goroutine
- Kill process if it doesn't fork within timeout
- Prevents goroutine accumulation on restart failures
- Location: cmd/bd/daemons.go:250-268
All changes follow Go best practices and maintain backward compatibility.
* Add feature request for .beads/README.md generation during init
Created bd-m7ge to automatically generate a promotional/documentation
README in the .beads directory when running 'bd init'. This will help
advertise Beads in open source repositories and provide quick reference
documentation for developers using AI coding agents.
The README will include:
- Brief explanation of Beads (AI-native issue tracking)
- Link to steveyegge/beads repository
- Quick reference of essential commands
- Compelling messaging to encourage adoption
* feat: add performance testing framework foundation
Implements foundation for comprehensive performance testing and user
diagnostics for beads databases at 10K-20K scale.
Components added:
- Fixture generator (internal/testutil/fixtures/) for realistic test data
* LargeSQLite/XLargeSQLite: 10K/20K issues with epic hierarchies
* LargeFromJSONL/XLargeFromJSONL: test JSONL import path
* Realistic cross-linked dependencies, labels, assignees
* Reproducible with seeded RNG
- User diagnostics (bd doctor --perf) for field performance data
* Collects platform info (OS, arch, Go/SQLite versions)
* Measures key operation timings (ready, list, show, search)
* Generates CPU profiles for bug reports
* Clean separation in cmd/bd/doctor/perf.go
Test data characteristics:
- 10% epics, 30% features, 60% tasks
- 4-level hierarchies (Epic → Feature → Task → Subtask)
- 20% cross-epic blocking dependencies
- Realistic status/priority/label distributions
Supports bd-l954 (Performance Testing Framework epic)
Closes bd-6ed8, bd-q59i
* perf: optimize GetReadyWork with compound index (20x speedup)
Add compound index on dependencies(depends_on_id, type, issue_id) to
eliminate performance bottleneck in GetReadyWork recursive CTE query.
Performance improvements (10K issue database):
- GetReadyWork: 752ms → 36.6ms (20.5x faster)
- Target: <50ms ✓ ACHIEVED
- 20K database: ~1500ms → 79.4ms (19x faster)
Benchmark infrastructure enhancements:
- Add dataset caching in /tmp/beads-bench-cache/ to avoid regenerating
10K-20K issues on every benchmark run (first run: ~2min, subsequent: <5s)
- Add progress logging during fixture generation (shows 10%, 20%... completion)
- Add database size logging (17.5 MB for 10K, 35.1 MB for 20K)
- Document rationale for only benchmarking large datasets (>10K issues)
- Add CPU/trace profiling with --profile flag for performance debugging
Schema changes:
- internal/storage/sqlite/schema.go: Add idx_dependencies_depends_on_type_issue
New files:
- internal/storage/sqlite/bench_helpers_test.go: Reusable benchmark setup with caching
- internal/storage/sqlite/sqlite_bench_test.go: Comprehensive benchmarks for critical operations
- Makefile: Convenient benchmark execution (make bench-quick, make bench)
Related:
- Resolves bd-5qim (optimize GetReadyWork performance)
- Builds on bd-6ed8 (fixture generator), bd-q59i (bd doctor --perf)
* perf: add WASM compilation cache to eliminate cold-start overhead
Configure wazero compilation cache for ncruces/go-sqlite3 to avoid
~220ms JIT compilation on every process start.
Cache configuration:
- Location: ~/.cache/beads/wasm/ (platform-specific via os.UserCacheDir)
- Automatic version management: wazero keys entries by its version
- Fallback: in-memory cache if directory creation fails
- No cleanup needed: old versions are harmless (~5-10MB each)
Performance impact:
- First run: ~220ms (populate cache)
- Subsequent runs: ~20ms (load from cache)
- Savings: ~200ms per cold start
Cache invalidation:
- Automatic when wazero version changes (upgrades use new cache dir)
- Manual cleanup: rm -rf ~/.cache/beads/wasm/ (safe to delete anytime)
This complements daemon mode:
- Daemon mode: eliminates startup cost by keeping process alive
- WASM cache: reduces startup cost for one-off commands or daemon restarts
Changes:
- internal/storage/sqlite/sqlite.go: Add init() with cache setup
* refactor: improve maintainability of performance testing code
Extract common patterns and eliminate duplication across benchmarks, fixture generation, and performance diagnostics. Replace magic numbers with explicit configuration to improve readability and make it easier to tune test parameters.
* docs: clarify profiling behavior and add missing documentation
Add explanatory comments for profiling setup to clarify why --profile
forces direct mode (captures actual database operations instead of RPC
overhead) and document the stopCPUProfile function's role in flushing
profile data to disk. Also fix gosec G104 linter warning by explicitly
ignoring Close() error during cleanup.
* fix: prevent bench-quick from running indefinitely
Added //go:build bench tags and skipped timeout-prone benchmarks to
prevent make bench-quick from running for hours.
Changes:
- Add //go:build bench tag to cycle_bench_test.go and compact_bench_test.go
- Skip Dense graph benchmarks (documented to timeout >120s)
- Fix compact benchmark prefix: bd- → bd (validation expects prefix without trailing dash)
Before: make bench-quick ran for 3.5+ hours (12,699s) before manual interrupt
After: make bench-quick completes in ~25 seconds
The Dense graph benchmarks are known to timeout and represent rare edge
cases that don't need optimization for typical workflows.
- Force single connection for all in-memory databases (including file::memory:)
- Close rows before executing statements in external_ref migration
- Prevents connection pool deadlock with MaxOpenConns(1)
- Fixes test failures in syncbranch_test.go
The schema initialization was trying to create an index on the external_ref
column before the migration that adds the column runs. This caused 'no such
column: external_ref' errors when opening very old databases (pre-0.17.5).
Solution: Move the index creation into the migration that adds the column.
Fixes#284
Amp-Thread-ID: https://ampcode.com/threads/T-2744d5a7-168f-4ef6-bcab-926db846de20
Co-authored-by: Amp <amp@ampcode.com>
- Implement comprehensive schema probe in sqlite.New() that verifies all
expected tables and columns after migrations
- Add retry logic: if probe fails, retry migrations once
- Return clear fatal error with missing schema elements if probe still fails
- Enhance daemon version gating: refuse RPC if client has newer minor version
- Improve checkVersionMismatch messaging: verify schema before claiming upgrade
- Add schema compatibility check to bd doctor command
- Add comprehensive tests for schema probing
This prevents the silent migration failure bug where:
1. Migrations fail silently
2. Database queries fail with 'no such column' errors
3. Import logic misinterprets as 'not found' and tries INSERT
4. Results in cryptic UNIQUE constraint errors
Fixes#262
Amp-Thread-ID: https://ampcode.com/threads/T-0d7ae2c0-9f12-4f9b-85d1-1291488af150
Co-authored-by: Amp <amp@ampcode.com>
