Commit Graph

16 Commits

Author SHA1 Message Date
Steve Yegge
599c40c703 Fix Windows and Nix test failures
Windows test fixes (bd-web8):
- Add sanitizeMetadataKey() to replace colons with underscores
- Windows absolute paths (e.g., C:\...) contain colons that conflict with
  metadata key separator ':'
- Update tests to use sanitized keys when checking metadata
- Tests now pass on Windows by auto-sanitizing path-based keys

Nix flake fixes (bd-8y1a):
- Update vendorHash in default.nix to match current Go module dependencies
- Hash mismatch was causing Nix build failures in CI

Test improvements:
- Rename TestUpdateExportMetadataInvalidKeySuffix to reflect new behavior
- Test now verifies sanitization instead of rejection
- All tests pass locally

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-22 10:59:40 -08:00
Steve Yegge
e816e91ecb Complete bd-ar2 P2 tasks: metadata, resurrection, and testing improvements
This commit addresses the remaining P2 tasks from bd-ar2 code review follow-up:

## Completed Tasks

### bd-ar2.4: Improve parent chain resurrection
- Modified `tryResurrectParentWithConn()` to recursively resurrect ancestor chain
- When resurrecting bd-root.1.2, now also resurrects bd-root.1 if missing
- Handles deeply nested hierarchies where intermediate parents are deleted
- All resurrection tests pass including new edge cases

### bd-ar2.5: Add error handling guidance
- Documented metadata update failure strategy in `updateExportMetadata()`
- Explained trade-off: warnings vs errors (safe, prevents data loss)
- Added user-facing message: "Next export may require running 'bd import' first"
- Clarifies that worst case is requiring import before next export

### bd-ar2.6: Document transaction boundaries
- Added comprehensive documentation for atomicity trade-offs
- Explained crash scenarios and recovery (bd import)
- Documented decision to defer defensive checks (Option 4) until needed
- No code changes - current approach is acceptable for now

### bd-ar2.12: Add metadata key validation
- Added keySuffix validation in `updateExportMetadata()` and `hasJSONLChanged()`
- Prevents ':' separator in keySuffix to avoid malformed metadata keys
- Documented metadata key format in function comments
- Single-repo: "last_import_hash", multi-repo: "last_import_hash:<repo_key>"

### bd-ar2.7: Add edge case tests for GetNextChildID resurrection
- TestGetNextChildID_ResurrectParent_NotInJSONL: parent not in history
- TestGetNextChildID_ResurrectParent_NoJSONL: missing JSONL file
- TestGetNextChildID_ResurrectParent_MalformedJSONL: invalid JSON lines
- TestGetNextChildID_ResurrectParentChain: deeply nested missing parents
- All tests pass, resurrection is robust against edge cases

## Files Changed
- cmd/bd/daemon_sync.go: Metadata validation, error handling docs
- cmd/bd/integrity.go: Added strings import, keySuffix validation
- internal/storage/sqlite/hash_ids.go: Improved resurrection comments
- internal/storage/sqlite/resurrection.go: Recursive ancestor resurrection
- internal/storage/sqlite/child_id_test.go: Added 4 new edge case tests

## Testing
All export, sync, metadata, and resurrection tests pass.
Edge cases properly handled: missing JSONL, malformed JSON, deep nesting.

## Remaining Tasks
- bd-ar2.8 (P3): Additional export metadata edge case tests (deferred)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 11:40:37 -05:00
Steve Yegge
cd8cb8b86a Fix bd-ar2 code review issues: metadata tracking and multi-repo support
This commit addresses critical code review findings from bd-dvd and bd-ymj fixes:

## Completed Tasks

### bd-ar2.1: Extract duplicated metadata update code
- Created `updateExportMetadata()` helper function
- Eliminated 22-line duplication between createExportFunc and createSyncFunc
- Single source of truth for metadata updates

### bd-ar2.2: Add multi-repo support to export metadata updates
- Added per-repo metadata key tracking with keySuffix parameter
- Both export and sync functions now update metadata for all repos

### bd-ar2.3: Fix tests to use actual daemon functions
- TestExportUpdatesMetadata now calls updateExportMetadata() directly
- Added TestUpdateExportMetadataMultiRepo() for multi-repo testing
- Fixed export_mtime_test.go tests to call updateExportMetadata()

### bd-ar2.9: Fix variable shadowing in GetNextChildID
- Changed `err` to `resurrectErr` to avoid shadowing
- Improves code clarity and passes linter checks

### bd-ar2.10: Fix hasJSONLChanged to support per-repo keys
- Updated hasJSONLChanged() to accept keySuffix parameter
- Reads metadata with correct per-repo keys
- All callers updated (validatePreExport, daemon import, sync command)

### bd-ar2.11: Use stable repo identifiers instead of paths
- Added getRepoKeyForPath() helper function
- Uses stable identifiers like ".", "../frontend" instead of absolute paths
- Metadata keys now portable across machines and clones
- Prevents orphaned metadata when repos are moved

## Files Changed
- cmd/bd/daemon_sync.go: Helper functions, metadata updates
- cmd/bd/integrity.go: hasJSONLChanged() with keySuffix support
- cmd/bd/sync.go: Updated to use getRepoKeyForPath()
- cmd/bd/*_test.go: Tests updated for new signatures
- internal/storage/sqlite/hash_ids.go: Fixed variable shadowing

## Testing
All export, sync, and integrity tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 11:28:25 -05:00
Steve Yegge
4c5f99c5bd Fix bd-dvd and bd-ymj: Parent resurrection and export metadata
Bug 1 (bd-dvd): GetNextChildID now attempts parent resurrection from JSONL
before failing. Added TryResurrectParent call to match CreateIssue behavior.

Bug 2 (bd-ymj): Export now updates last_import_hash metadata to prevent
'JSONL content has changed' errors on subsequent exports.

Files changed:
- internal/storage/sqlite/hash_ids.go: Add resurrection attempt
- cmd/bd/daemon_sync.go: Add metadata updates after export
- Tests added for both fixes
- Fixed pre-existing bug in integrity_content_test.go

Follow-up work tracked in epic bd-ar2 (9 issues for improvements).

Fixes GH #334
2025-11-21 10:29:30 -05:00
Steve Yegge
0020eb490c Fix bd-khnb: Prevent auto-import from resurrecting deleted issues
Replace mtime-based staleness detection with content-based (SHA256 hash) to prevent
git operations from resurrecting deleted issues.

**Problem:**
Auto-import used file modification time to detect if JSONL was "newer" than database.
Git operations (checkout, merge, pull) restore old files with recent mtimes, causing
auto-import to load stale data over current database state, resurrecting deleted issues.

**Solution:**
- Added computeJSONLHash() to compute SHA256 of JSONL content
- Added hasJSONLChanged() with two-tier check:
  1. Fast-path: Check mtime first (99% of checks are instant)
  2. Slow-path: Compute hash only if mtime changed (catches git operations)
- Store metadata: last_import_hash, last_import_mtime, last_import_time
- Updated auto-import in daemon_sync.go to use content-based check
- Updated validatePreExport to use content-based check (bd-xwo)
- Graceful degradation: metadata failures are non-fatal warnings

**Changes:**
- cmd/bd/integrity.go: Add computeJSONLHash(), hasJSONLChanged()
- cmd/bd/integrity_test.go: Add comprehensive tests for new functions
- cmd/bd/import.go: Update metadata after import
- cmd/bd/sync.go: Use hasJSONLChanged() instead of isJSONLNewer()
- cmd/bd/daemon_sync.go: Use hasJSONLChanged() in auto-import

**Testing:**
- Unit tests pass (TestHasJSONLChanged with 7 scenarios)
- Integration test passes (test_bd_khnb_fix.sh)
- Verified git resurrection scenario prevented

Fixes: bd-khnb
Related: bd-3bg, bd-xwo, bd-39o, bd-56p, bd-m8t, bd-rfj, bd-t5o

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 22:00:20 -05:00
Steve Yegge
57253f93a3 Context propagation with graceful cancellation (bd-rtp, bd-yb8, bd-2o2)
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.

Key changes:

1. Signal-aware contexts (bd-rtp):
   - Added rootCtx/rootCancel in main.go using signal.NotifyContext()
   - Set up in PersistentPreRun, cancelled in PersistentPostRun
   - Daemon uses same pattern in runDaemonLoop()
   - Handles SIGINT/SIGTERM for graceful shutdown

2. Context propagation (bd-yb8):
   - All commands now use rootCtx instead of context.Background()
   - sqlite.New() receives context for cancellable operations
   - Database operations respect context cancellation
   - Storage layer propagates context through all queries

3. Cancellation tests (bd-2o2):
   - Added import_cancellation_test.go with comprehensive tests
   - Added export cancellation test in export_test.go
   - Tests verify database integrity after cancellation
   - All cancellation tests passing

Fixes applied during review:
   - Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
   - Fixed test context contamination (reset rootCtx in test cleanup)
   - Fixed export tests missing context setup

Impact:
   - Pressing Ctrl+C during import/export now cancels gracefully
   - No database corruption or hanging transactions
   - Clean shutdown of all operations

Tested:
   - go build ./cmd/bd ✓
   - go test ./cmd/bd -run TestImportCancellation ✓
   - go test ./cmd/bd -run TestExportCommand ✓
   - Manual Ctrl+C testing verified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 21:57:23 -05:00
Charles P. Cross
04a1996fd9 fix: Fix daemon export leaving JSONL newer than database (issues #301, #321)
After daemon auto-export, JSONL mtime could be newer than database mtime
due to SQLite WAL mode not updating beads.db until checkpoint. This caused
validatePreExport to incorrectly block subsequent exports with "JSONL is
newer than database" error, leading to daemon shutdown.

Solution: Call TouchDatabaseFile after all export operations to ensure
database mtime >= JSONL mtime. This prevents false positives in validation
2025-11-19 05:06:12 -05:00
Steve Yegge
7afb1439f2 Cache getMultiRepoJSONLPaths() to avoid redundant calls (bd-we4p)
- Call getMultiRepoJSONLPaths() once at sync start instead of 3 times
- Eliminates redundant config lookups and path constructions
- Performance optimization for multi-repo workflows
2025-11-06 19:41:38 -08:00
Steve Yegge
47185830bb Fix bd-4oob: Add multi-repo deletion tracking support
- Added getMultiRepoJSONLPaths() helper to enumerate all JSONL files
- Updated daemon_sync.go to capture/update snapshots for each repo
- Added TestMultiRepoDeletionTracking to verify fix
- Added TestMultiRepoSnapshotIsolation for isolation guarantees

Previously, deletion tracking only worked for single-repo mode because
snapshot operations were hardcoded to the primary JSONL path. This fix
ensures snapshots are managed per-repository in multi-repo mode.
2025-11-06 19:21:11 -08:00
Steve Yegge
e5a6c05e38 Fix P1 deletion tracking bugs (bd-rbxi)
- bd-nqes: Made snapshot capture mandatory with fail-fast
- bd-mn9p: Added semantic JSON comparison (jsonEquals)
- bd-2ifg: Collect deletion errors and fail operation
- bd-8ayj: Atomic file ops with PID-specific temp files
- bd-aewm: Added defer cleanup for .merged temp file

All tests pass.

Amp-Thread-ID: https://ampcode.com/threads/T-5e744954-8a08-4697-960e-5f2a88d50c54
Co-authored-by: Amp <amp@ampcode.com>
2025-11-06 18:47:06 -08:00
Steve Yegge
708a81c491 Fix bd-hv01: Implement deletion tracking for multi-workspace sync
- Add 3-way merge deletion tracking using snapshot files
- Create .beads/beads.base.jsonl and .beads/beads.left.jsonl snapshots
- Integrate into both sync.go and daemon_sync.go
- Add comprehensive test suite in deletion_tracking_test.go
- Update .gitignore to exclude snapshot files

This fixes the resurrection bug where deleted issues come back after
multi-workspace git sync. Uses the beads-merge 3-way merge logic to
detect and apply deletions correctly.
2025-11-06 17:52:37 -08:00
Steve Yegge
584c266684 Reorganize project structure: move Go files to internal/beads, docs to docs/
Amp-Thread-ID: https://ampcode.com/threads/T-7a71671d-dd5c-4c7c-b557-fa427fceb04f
Co-authored-by: Amp <amp@ampcode.com>
2025-11-05 21:04:00 -08:00
Steve Yegge
67710e4a0c Add multi-repo CLI commands and integrate with daemon sync 2025-11-04 23:12:41 -08:00
Steve Yegge
86c645603e Fix all gosec, misspell, and unparam linter errors
- Add #nosec directives with explanations for all gosec warnings in worktree operations
- Tighten directory permissions from 0755 to 0750 for better security
- Fix misspellings: archaeological -> archeological, cancelled -> canceled
- Remove unused jsonlPath parameter from syncBranchCommitAndPush
- Change branchExists to return bool instead of (bool, error) - error was never used

All changes maintain backward compatibility and improve code quality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 20:06:05 -08:00
Steve Yegge
712fb772fe Complete sync branch daemon tests (bd-7bd2, bd-502e)
- Add daemon_sync_branch.go with syncBranchCommitAndPush/Pull functions
- Add daemon_sync_branch_test.go with 7 comprehensive tests
- All tests passing: NotConfigured, Success, NoChanges, WorktreeHealthCheck, Pull, EndToEnd
- Key fix: initMainBranch must run BEFORE creating issues/JSONL
- Close bd-7bd2 and bd-502e

Amp-Thread-ID: https://ampcode.com/threads/T-e3d7ba22-99d1-4210-a6db-1dcc3bdd622b
Co-authored-by: Amp <amp@ampcode.com>
2025-11-02 16:40:21 -08:00
Steve Yegge
eb00ab8005 Refactor daemon.go for testability and maintainability (bd-2b34)
- Split 1567-line daemon.go into 5 focused modules
- daemon_config.go (122 lines): config/path resolution
- daemon_lifecycle.go (451 lines): start/stop/status/health/metrics
- daemon_sync.go (510 lines): export/import/sync logic
- daemon_server.go (115 lines): RPC server setup
- daemon_logger.go (43 lines): logging utilities
- Reduced main daemon.go to 389 lines (75% reduction)
- All tests pass, improved code organization

Amp-Thread-ID: https://ampcode.com/threads/T-7504c501-f962-4b82-a6d9-8e33f547757d
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 19:20:01 -07:00