Update withTx to use BEGIN IMMEDIATE with exponential backoff retry
on SQLITE_BUSY errors. This prevents "database is locked" failures
during concurrent operations (daemon + CLI, multi-agent workflows).
Changes:
- withTx now uses beginImmediateWithRetry (same pattern as RunInTransaction)
- Add dbExecutor interface for helper functions that work with both
*sql.Tx and *sql.Conn
- Update all withTx callers to use *sql.Conn
- Refactor DeleteIssue to use withTx (fixes the specific error in auto-import)
- Update markIssuesDirtyTx to accept dbExecutor interface
Affected paths:
- MarkIssuesDirty, ClearDirtyIssuesByID (dirty.go)
- AddDependency, RemoveDependency (dependencies.go)
- executeLabelOperation (labels.go)
- AddComment (events.go)
- ApplyCompaction (compact.go)
- DeleteIssue (queries.go)
Note: Some direct BeginTx calls in queries.go (CloseIssue, UpdateIssue,
ReopenIssue, DeleteIssues) still use the old pattern and could be
refactored in a follow-up.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add missing reconnectMu.RLock() protection to storage methods that were
vulnerable to the same race condition fixed in GH#607. The FreshnessChecker
can trigger reconnect() which closes s.db while queries are in flight,
causing "database is closed" errors during daemon export operations.
Protected methods:
- labels.go: GetLabelsForIssues (GetLabels intentionally unprotected - called from GetIssue which holds lock)
- comments.go: GetIssueComments, GetCommentsForIssues
- dependencies.go: GetDependencyCounts, GetDependencyRecords, GetAllDependencyRecords, GetDependencyTree, loadDependencyGraph
- config.go: SetConfig, GetConfig, GetAllConfig, DeleteConfig, SetMetadata, GetMetadata
- dirty.go: MarkIssueDirty, GetDirtyIssues, GetDirtyIssueHash, GetDirtyIssueCount
- events.go: GetEvents, GetStatistics, GetMoleculeProgress
- hash.go: All hash methods
- hash_ids.go: GetNextChildID, ensureChildCounterUpdated (getNextChildNumber unprotected - called internally)
Internal helpers called from already-locked contexts intentionally omit
RLock to avoid deadlock (Go's RWMutex doesn't support recursive locking).
Fixes: bd-vx7fp
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Executed-By: beads/crew/dave
Rig: beads
Role: crew
Remove ClearDirtyIssues() which had a race condition that could lose
dirty issues if export failed partway through. All callers now use
ClearDirtyIssuesByID() which only clears specific exported issues.
- Remove from Storage interface
- Remove from SQLite and Memory implementations
- Update 6 test call sites to use ClearDirtyIssuesByID()
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Created internal/storage/sqlite/errors.go with:
- Sentinel errors: ErrNotFound, ErrInvalidID, ErrConflict, ErrCycle
- wrapDBError helpers that auto-convert sql.ErrNoRows to ErrNotFound
- Type-safe error checking with errors.Is() compatibility
Updated error handling across storage layer:
- dirty.go: Added context to error returns, converted sql.ErrNoRows checks
- util.go: Updated withTx to use wrapDBError
- batch_ops.go: Added context wrapping to batch operations
- dependencies.go: Wrapped errors from markIssuesDirtyTx calls
- ids.go: Added error wrapping for ID validation
Also restored sqlite.go that was accidentally deleted in previous commit.
All tests pass. Provides consistent error wrapping with operation context
for better debugging.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace duplicated dirty-marking logic in AddDependency and
RemoveDependency with a new markIssuesDirtyTx helper function.
This improves code maintainability and ensures consistent behavior.
The Problem:
- AddDependency and RemoveDependency had ~20 lines of duplicated code
- Each manually prepared statements and marked issues dirty
- Violation of DRY principle
- Pattern was fragile if preparation failed
The Fix:
- Created markIssuesDirtyTx() helper in dirty.go
- Takes existing transaction and issue IDs
- Both functions now use: markIssuesDirtyTx(ctx, tx, []string{id1, id2})
- Reduced from 20 lines to 3 lines per function
Benefits:
- Eliminates code duplication (DRY)
- Single source of truth for transaction-based dirty marking
- More readable and maintainable
- Easier to modify behavior in future
- Consistent error messages
Changes:
- internal/storage/sqlite/dirty.go:129-154
* Add markIssuesDirtyTx() helper function
- internal/storage/sqlite/dependencies.go:115-117, 147-149
* Replace duplicated code with helper call in both functions
Testing:
- All existing tests pass ✓
- Verified both issues marked dirty with same timestamp ✓
- Dependency add/remove works correctly ✓
Impact:
- Cleaner, more maintainable codebase
- No functional changes, pure refactor
- Foundation for future improvements
Closes bd-56
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix critical TOCTOU bug where concurrent operations could lose dirty
issue tracking, causing data loss in incremental exports. Also fixes
bug where export with filters would incorrectly clear all dirty issues.
The Problem:
1. GetDirtyIssues() returns [bd-1, bd-2]
2. Concurrent CRUD marks bd-3 dirty
3. Export writes bd-1, bd-2
4. ClearDirtyIssues() deletes ALL (including bd-3)
5. Result: bd-3 never gets exported!
The Fix:
- Add ClearDirtyIssuesByID() that only clears specific issue IDs
- Track which issues were actually exported
- Clear only those specific IDs, not all dirty issues
- Fixes both race condition and filter export bug
Changes:
- internal/storage/sqlite/dirty.go:
* Add ClearDirtyIssuesByID() method
* Add warning to ClearDirtyIssues() about race condition
- internal/storage/storage.go:
* Add ClearDirtyIssuesByID to interface
- cmd/bd/main.go:
* Update auto-flush to use ClearDirtyIssuesByID()
- cmd/bd/export.go:
* Track exported issue IDs
* Use ClearDirtyIssuesByID() instead of ClearDirtyIssues()
Testing:
- Created test-1, test-2, test-3 (all dirty)
- Updated test-2 to in_progress
- Exported with --status open filter (exports only test-1, test-3)
- Verified only test-2 remains dirty ✓
- All existing tests pass ✓
Impact:
- Race condition eliminated - concurrent operations are safe
- Export with filters now works correctly
- No data loss from competing writes
Closes bd-52, bd-53
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Optimize auto-flush by tracking which issues have changed instead of
exporting the entire database on every flush. For large projects with
1000+ issues, this provides significant performance improvements.
Changes:
- Add dirty_issues table to schema with issue_id and marked_at columns
- Implement dirty tracking functions in new dirty.go file:
* MarkIssueDirty() - Mark single issue as needing export
* MarkIssuesDirty() - Batch mark multiple issues efficiently
* GetDirtyIssues() - Query which issues need export
* ClearDirtyIssues() - Clear tracking after successful export
* GetDirtyIssueCount() - Monitor dirty issue count
- Update all CRUD operations to mark affected issues as dirty:
* CreateIssue, UpdateIssue, DeleteIssue
* AddDependency, RemoveDependency (marks both issues)
* AddLabel, RemoveLabel, AddEvent
- Modify export to support incremental mode:
* Add --incremental flag to export only dirty issues
* Used by auto-flush for performance
* Full export still available without flag
- Add Storage interface methods for dirty tracking
Performance impact: With incremental export, large databases only write
changed issues instead of regenerating entire JSONL file on every
auto-flush.
Closes bd-39
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
