Implements three quick fixes for users stuck in sandboxed environments
(e.g., Codex) where daemon cannot be stopped:
1. **--force flag for bd import**
- Forces metadata update even when DB is synced with JSONL
- Fixes stuck state caused by stale daemon cache
- Shows: "Metadata updated (database already in sync with JSONL)"
2. **--allow-stale global flag**
- Emergency escape hatch to bypass staleness check
- Shows warning: "⚠️ Staleness check skipped (--allow-stale)"
- Allows operations on potentially stale data
3. **Improved error message**
- Added sandbox-specific guidance to staleness error
- Suggests --sandbox, --force, and --allow-stale flags
- Provides clear fix steps for different scenarios
Also fixed:
- Removed unused import in cmd/bd/duplicates_test.go
Follow-up work filed:
- bd-u3t: Phase 2 - Sandbox auto-detection
- bd-e0o: Phase 3 - Daemon robustness enhancements
- bd-9nw: Documentation updates
Fixes#353 (Phase 1)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace mtime-based staleness detection with content-based (SHA256 hash) to prevent
git operations from resurrecting deleted issues.
**Problem:**
Auto-import used file modification time to detect if JSONL was "newer" than database.
Git operations (checkout, merge, pull) restore old files with recent mtimes, causing
auto-import to load stale data over current database state, resurrecting deleted issues.
**Solution:**
- Added computeJSONLHash() to compute SHA256 of JSONL content
- Added hasJSONLChanged() with two-tier check:
1. Fast-path: Check mtime first (99% of checks are instant)
2. Slow-path: Compute hash only if mtime changed (catches git operations)
- Store metadata: last_import_hash, last_import_mtime, last_import_time
- Updated auto-import in daemon_sync.go to use content-based check
- Updated validatePreExport to use content-based check (bd-xwo)
- Graceful degradation: metadata failures are non-fatal warnings
**Changes:**
- cmd/bd/integrity.go: Add computeJSONLHash(), hasJSONLChanged()
- cmd/bd/integrity_test.go: Add comprehensive tests for new functions
- cmd/bd/import.go: Update metadata after import
- cmd/bd/sync.go: Use hasJSONLChanged() instead of isJSONLNewer()
- cmd/bd/daemon_sync.go: Use hasJSONLChanged() in auto-import
**Testing:**
- Unit tests pass (TestHasJSONLChanged with 7 scenarios)
- Integration test passes (test_bd_khnb_fix.sh)
- Verified git resurrection scenario prevented
Fixes: bd-khnb
Related: bd-3bg, bd-xwo, bd-39o, bd-56p, bd-m8t, bd-rfj, bd-t5o
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.
Key changes:
1. Signal-aware contexts (bd-rtp):
- Added rootCtx/rootCancel in main.go using signal.NotifyContext()
- Set up in PersistentPreRun, cancelled in PersistentPostRun
- Daemon uses same pattern in runDaemonLoop()
- Handles SIGINT/SIGTERM for graceful shutdown
2. Context propagation (bd-yb8):
- All commands now use rootCtx instead of context.Background()
- sqlite.New() receives context for cancellable operations
- Database operations respect context cancellation
- Storage layer propagates context through all queries
3. Cancellation tests (bd-2o2):
- Added import_cancellation_test.go with comprehensive tests
- Added export cancellation test in export_test.go
- Tests verify database integrity after cancellation
- All cancellation tests passing
Fixes applied during review:
- Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
- Fixed test context contamination (reset rootCtx in test cleanup)
- Fixed export tests missing context setup
Impact:
- Pressing Ctrl+C during import/export now cancels gracefully
- No database corruption or hanging transactions
- Clean shutdown of all operations
Tested:
- go build ./cmd/bd ✓
- go test ./cmd/bd -run TestImportCancellation ✓
- go test ./cmd/bd -run TestExportCommand ✓
- Manual Ctrl+C testing verified
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
After daemon auto-export, JSONL mtime could be newer than database mtime
due to SQLite WAL mode not updating beads.db until checkpoint. This caused
validatePreExport to incorrectly block subsequent exports with "JSONL is
newer than database" error, leading to daemon shutdown.
Solution: Call TouchDatabaseFile after all export operations to ensure
database mtime >= JSONL mtime. This prevents false positives in validation
When bd import finds no changes (0 created, 0 updated), it now updates
the database modification timestamp to signal sync validation passed.
This fixes 'bd sync' refusing to export with 'JSONL is newer than database'
error that occurs after git pull updates JSONL mtime without content changes.
Root cause: Import validated DB/JSONL are in sync but didn't update DB mtime,
causing timestamp-based sync validation to perpetually fail.
Amp-Thread-ID: https://ampcode.com/threads/T-560b9c11-e368-45e6-b1e3-512b6d6010a1
Co-authored-by: Amp <amp@ampcode.com>
- import.go: Check raw bytes before JSON decoding using bytes.HasPrefix
- validate.go: Use bytes.Split and bytes.HasPrefix on raw data
- Added regression test TestAutoImportConflictMarkerFalsePositive
- Verified with vc-85 issue that triggered the bug
Amp-Thread-ID: https://ampcode.com/threads/T-3f81e22a-14b9-435b-8932-5641aadb7d31
Co-authored-by: Amp <amp@ampcode.com>
When 'bd sync --import-only' completes, it imports JSONL changes into
the database but doesn't update the database file's modification time.
This causes 'bd doctor' to incorrectly warn that 'JSONL is newer than
database' even when they're in sync.
Root cause: SQLite in WAL mode writes to beads.db-wal; the main beads.db
mtime often doesn't change until a checkpoint. bd doctor compares JSONL
mtime to beads.db mtime, so it can misfire without an mtime bump.
The fix adds touchDatabaseFile() that:
- Only runs when import actually made changes (not dry-run, not unchanged)
- Sets DB mtime to max(JSONL mtime, now) + 1ns to handle clock skew
- Is best-effort (logs warning on failure, doesn't fail import)
- Includes tests for basic touch and clock skew scenarios
Fixes: bd-g3ey
When git pull encounters merge conflicts in .beads/beads.jsonl, the
post-merge hook runs 'bd sync --import-only' which previously failed
with an error message pointing users to manual resolution.
This commit adds automatic 3-way merge resolution as a fallback safety
net that works alongside the git merge driver.
Changes:
- Modified conflict detection in import.go to attempt automatic merge
- Added attemptAutoMerge() function that:
- Extracts git conflict stages (:1 base, :2 ours, :3 theirs)
- Invokes 'bd merge' command for intelligent field-level merging
- Writes merged result back and auto-stages the file
- Restarts import with merged JSONL on success
- Falls back to manual resolution instructions only if auto-merge fails
Defense-in-depth approach:
1. Primary: git merge driver prevents most conflicts during merge
2. Fallback: import auto-merge handles any that slip through
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Agents were failing with 'bd import' after nuking the database.
Now bd import automatically:
- Creates .beads/ directory if missing
- Detects prefix from imported issues (analyzes JSONL)
- Falls back to directory name if no issues
- Initializes database before proceeding
This fixes the common agent workflow: rm .beads/beads.db && bd import
Amp-Thread-ID: https://ampcode.com/threads/T-3ace45ce-3cd6-46ae-a201-39c3d7f48f0b
Co-authored-by: Amp <amp@ampcode.com>
- Detect uncommitted changes in .beads/issues.jsonl
- Warn users when database matches working tree but differs from git HEAD
- Clarify import status messages (working tree vs git sync)
- Add comprehensive tests for dirty working tree scenarios
- Prevents false confidence about sync status
Amp-Thread-ID: https://ampcode.com/threads/T-5a0f1045-a690-42ef-8bfc-f8cf30ee4084
Co-authored-by: Amp <amp@ampcode.com>
Root cause: Import and export commands tried to open the database directly
while the daemon already held the lock, causing indefinite blocking.
Solution: Both commands now explicitly close the daemon connection before
opening direct database access, avoiding SQLite lock contention.
Changes:
- import.go: Close daemon connection and open direct SQLite connection
- export.go: Close daemon connection before direct access
- Added debug logging to help diagnose similar issues
Tests: Existing TestImport and TestExport tests pass
Fixes GH-234 by providing automatic resolution for duplicate external_ref
values instead of forcing manual JSONL editing.
Changes:
- Add ClearDuplicateExternalRefs option to importer.Options
- Modify validateNoDuplicateExternalRefs to clear duplicates when enabled
- Keep first occurrence, clear rest when flag is set
- Enhanced error message to suggest the flag
- Add comprehensive tests for the new behavior
Usage: bd import -i issues.jsonl --clear-duplicate-external-refs
Amp-Thread-ID: https://ampcode.com/threads/T-932dcf45-76f2-4994-9b5c-a6eb20a86036
Co-authored-by: Amp <amp@ampcode.com>
- Changed bd import to call flushToJSONL() instead of markDirtyAndScheduleFlush()
- Ensures daemon FileWatcher detects changes within ~500ms instead of up to 30s
- Fixes race condition where MCP daemon served stale data after CLI import
Hash-based IDs make collision resolution unnecessary. The flag was
already non-functional (handleCollisions returns error on collision
regardless of flag value).
Removed:
- --resolve-collisions flag from bd import
- ResolveCollisions field from ImportOptions and importer.Options
- All references in daemon, auto-import, and tests
- Updated error messages to reflect hash IDs don't collide
All import tests pass.
Amp-Thread-ID: https://ampcode.com/threads/T-47dfa0cc-bb71-4467-ac86-f0966a7c5d58
Co-authored-by: Amp <amp@ampcode.com>
- New 'bd duplicates' command finds content duplicates across database
- Groups by content hash (title, description, design, acceptance criteria)
- Chooses merge target by reference count or smallest ID
- Supports --auto-merge and --dry-run flags
- Added --dedupe-after flag to 'bd import' for post-import detection
- Comprehensive test coverage for duplicate detection
- Updated AGENTS.md and ADVANCED.md with usage examples
Amp-Thread-ID: https://ampcode.com/threads/T-6f99566f-c979-43ed-bd8f-5aa38b0f6191
Co-authored-by: Amp <amp@ampcode.com>
When importing JSONL that matches the database exactly, import was
reporting '0 created, 0 updated' which was confusing. Now it properly
tracks and reports unchanged issues.
Changes:
- Added Unchanged field to ImportResult
- Track unchanged issues separately from skipped/updated
- Display unchanged count in import summary
- Updated dry-run output to show unchanged count
- Added test to verify correct reporting behavior
Example output: 'Import complete: 0 created, 0 updated, 88 unchanged'
Amp-Thread-ID: https://ampcode.com/threads/T-5dd4843e-9ce3-4fe0-9658-d2227b0a2e4e
Co-authored-by: Amp <amp@ampcode.com>
- Detects prefix mismatches during import
- Shows warning with mismatched prefixes and counts
- Added --rename-on-import flag to automatically fix prefixes
- Auto-import is lenient about prefixes (doesn't enforce)
- All tests passing
- Implements bd stale command to show issues with execution_state where executor is dead/stopped
- Adds --release flag to automatically release orphaned issues
- Adds --threshold flag to customize heartbeat staleness threshold (default: 300s/5min)
- Handles missing executor instances (LEFT JOIN) for cases where executor was deleted
- Adds QueryContext and BeginTx helper methods to SQLiteStorage for advanced queries
- Fixes ExternalRef comparison bug in import_shared.go (pointer vs string)
- Removes unused imports in import.go
Resolves vc-124
The skipUpdate flag should only apply to existing DB issues, not to
duplicates within the import batch. Batch duplicates always use
last-one-wins semantics.
- Replace CreateIssue loop with single CreateIssues batch call
- Add in-memory de-duplication for duplicate IDs (last one wins)
- 5-15x faster for bulk imports
- All tests pass
- Add closed_at field to Issue type with JSON marshaling
- Implement closed_at timestamp in SQLite storage layer
- Update import/export to handle closed_at field
- Add comprehensive tests for closed_at functionality
- Maintain backward compatibility with existing databases
Amp-Thread-ID: https://ampcode.com/threads/T-f3a7799b-f91e-4432-a690-aae0aed364b3
Co-authored-by: Amp <amp@ampcode.com>
Add nullable external_ref TEXT field to link bd issues with external
systems like GitHub Issues, Jira, etc. Includes automatic schema
migration for backward compatibility.
Changes:
- Added external_ref column to issues table with feature-based migration
- Updated Issue struct with ExternalRef *string field
- Added --external-ref flag to bd create and bd update commands
- Updated all SQL queries across the codebase to include external_ref:
- GetIssue, CreateIssue, UpdateIssue, SearchIssues
- GetDependencies, GetDependents, GetDependencyTree
- GetReadyWork, GetBlockedIssues, GetIssuesByLabel
- Added external_ref handling in import/export logic
- Follows existing patterns for nullable fields (sql.NullString)
This enables tracking relationships between bd issues and external
systems without requiring changes to existing databases or JSONL files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses all critical follow-up issues identified in the
code review of PR #8 (atomic counter implementation).
## bd-64: Fix SyncAllCounters performance bottleneck (P0)
- Replace SyncAllCounters() on every CreateIssue with lazy initialization
- Add ensureCounterInitialized() that only scans prefix-specific issues on first use
- Performance improvement: O(n) full table scan → O(1) for subsequent creates
- Add comprehensive tests in lazy_init_test.go
## bd-65: Add migration for issue_counters table (P1)
- Add migrateIssueCountersTable() similar to migrateDirtyIssuesTable()
- Checks if table is empty and syncs from existing issues on first open
- Handles both fresh databases and migrations from old databases
- Add comprehensive tests in migration_test.go (3 scenarios)
## bd-66: Make import counter sync failure fatal (P1)
- Change SyncAllCounters() failure from warning to fatal error in import
- Prevents ID collisions when counter sync fails
- Data integrity > convenience
## bd-67: Update test comments (P2)
- Update TestMultiProcessIDGeneration comments to reflect fix is in place
- Change "With the bug, we expect errors" → "After the fix, all should succeed"
All tests pass. Atomic counter implementation is now production-ready.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When importing issues with explicit high IDs (e.g., bd-100), the
issue_counters table wasn't being updated. This caused the next
auto-generated issue to collide with existing IDs (bd-4 instead of bd-101).
Changes:
- Add SyncAllCounters() to scan all issues and update counters atomically
- Add SyncCounterForPrefix() for granular counter synchronization
- Call SyncAllCounters() in import command after creating issues
- Add comprehensive tests for counter sync functionality
- Update TestImportCounterSyncAfterHighID to verify fix
The fix uses a single efficient SQL query to prevent ID collisions
with subsequently auto-generated issues.
Fixed three critical issues identified in code review:
1. Race condition with store access: Added storeMutex and storeActive
flag to prevent background flush goroutine from accessing closed
store. Background timer now safely checks if store is active before
attempting flush operations.
2. Missing auto-flush in import: Added markDirtyAndScheduleFlush()
call after import completes, ensuring imported issues sync to JSONL.
3. Timer cleanup: Explicitly set flushTimer to nil after Stop() to
prevent resource leaks.
Testing confirmed all fixes working:
- Debounced flush triggers after 5 seconds of inactivity
- Immediate flush on process exit works correctly
- Import operations now trigger auto-flush
- No race conditions detected
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed critical issues identified in code review:
- Fixed invalid Go version (1.25.2 → 1.21) in go.mod
- Fixed unchecked error in import.go JSON unmarshaling
- Fixed unchecked error returns in test cleanup (export_import_test.go, import_collision_test.go)
- Removed duplicate test code in dependencies_test.go via helper function
Added release infrastructure:
- Added 'bd version' command with JSON output support
- Created comprehensive CHANGELOG.md following Keep a Changelog format
- Updated README.md with clear alpha status warnings
All tests passing. Ready for public repository opening.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements --resolve-collisions flag for import command to safely handle ID
collisions during branch merges. When enabled, colliding issues are remapped
to new IDs and all text references and dependencies are automatically updated.
Also adds comprehensive tests, branch-merge example, and documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added dependency support to export/import workflow:
Changes:
- Added GetDependencyRecords() to Storage interface to get raw dependency records
- Extended Issue struct with Dependencies field (omitempty for backward compat)
- Modified export.go to populate dependencies for each issue
- Modified import.go to process dependencies in second pass after all issues exist
- All tests pass
Benefits:
- JSONL is now self-contained with full dependency information
- Enables proper collision resolution in future (bd-12+)
- Idempotent imports: existing dependencies are not duplicated
- Forward references handled: dependencies created after all issues exist
Example output:
{
"id": "bd-10",
"title": "...",
"dependencies": [{
"issue_id": "bd-10",
"depends_on_id": "bd-9",
"type": "parent-child",
"created_at": "...",
"created_by": "stevey"
}]
}
Closes bd-10
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Major improvements to code quality, documentation, and CI:
Code Quality:
- Add golangci-lint configuration with 13 linters
- Fix unchecked error returns in export/import/init
- Refactor duplicate scanIssues code
- Add package comments for all packages
- Add const block comments for exported constants
- Configure errcheck to allow idiomatic defer patterns
Documentation:
- Add comprehensive CONTRIBUTING.md with setup, testing, and workflow
- Fix QUICKSTART.md binary name references (beads → bd)
- Correct default database path documentation
CI/CD:
- Add GitHub Actions workflow for tests and linting
- Enable race detection and coverage reporting
- Automated quality checks on all PRs
All tests passing. Lint issues reduced from 117 to 103 (remaining are
idiomatic patterns and test code). Ready for open-source release.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This is a fundamental architectural shift from binary SQLite to JSONL as
the source of truth for git workflows.
## New Features
- `bd export --format=jsonl` - Export issues to JSON Lines format
- `bd import` - Import issues from JSONL (create new, update existing)
- `--skip-existing` flag for import to only create new issues
## Architecture Change
**Before:** Binary SQLite database committed to git
**After:** JSONL text files as source of truth, SQLite as ephemeral cache
Benefits:
- Git-friendly text format with clean diffs
- AI-resolvable merge conflicts (append-only is 95% conflict-free)
- Human-readable issue tracking in git
- No binary merge conflicts
## Documentation
- Updated README with JSONL-first workflow and git hooks
- Added TEXT_FORMATS.md analyzing JSONL vs CSV vs binary
- Updated GIT_WORKFLOW.md with historical context
- .gitignore now excludes *.db, includes .beads/*.jsonl
## Implementation Details
- Export sorts issues by ID for consistent diffs
- Import handles both creates and updates atomically
- Proper handling of pointer fields (EstimatedMinutes)
- All tests passing
## Breaking Changes
- Database files (*.db) should now be gitignored
- Use export/import workflow for git collaboration
- Git hooks recommended for automation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
