Commit Graph

533 Commits

Author SHA1 Message Date
Steve Yegge
2d53ba347a bd sync: 2025-10-29 23:47:57 2025-10-29 23:47:57 -07:00
Steve Yegge
0b51f8c865 bd sync: 2025-10-29 23:20:22 2025-10-29 23:20:22 -07:00
Steve Yegge
5aed853646 bd sync: 2025-10-29 23:14:46 2025-10-29 23:14:46 -07:00
Steve Yegge
ea38485718 Update issues 2025-10-29 23:00:42 -07:00
Steve Yegge
24fcec03f4 bd sync: 2025-10-29 22:55:15 2025-10-29 22:55:15 -07:00
Steve Yegge
9ba77e981f bd sync: 2025-10-29 21:57:23 2025-10-29 21:57:23 -07:00
Steve Yegge
c34b93fa1a Fix bd-160: Implement JSONL integrity validation and prevent export deduplication data loss
## Problem
Export deduplication feature broke when JSONL and export_hashes diverged
(e.g., after git pull/reset). This caused exports to skip issues that
weren't actually in the file, leading to silent data loss.

## Solution
1. JSONL integrity validation before every export
   - Store JSONL file hash after export
   - Validate hash before export, clear export_hashes if mismatch
   - Automatically recovers from git operations changing JSONL

2. Clear export_hashes on all imports
   - Prevents stale hashes from causing future export failures
   - Import operations invalidate export_hashes state

3. Add Storage interface methods:
   - GetJSONLFileHash/SetJSONLFileHash for integrity tracking
   - ClearAllExportHashes for recovery

## Tests Added
- TestJSONLIntegrityValidation: Unit tests for validation logic
- TestImportClearsExportHashes: Verifies imports clear hashes
- TestExportIntegrityAfterJSONLTruncation: Simulates git reset (would have caught bd-160)
- TestExportIntegrityAfterJSONLDeletion: Tests recovery from file deletion
- TestMultipleExportsStayConsistent: Tests repeated export integrity

## Follow-up
Created bd-179 epic for remaining integration test gaps (multi-repo sync,
daemon auto-sync, corruption recovery tests).

Closes bd-160
2025-10-29 21:57:15 -07:00
Steve Yegge
d5400dcdeb bd sync: 2025-10-29 21:37:39 2025-10-29 21:37:39 -07:00
Steve Yegge
c6476fe2da bd sync: 2025-10-29 21:30:19 2025-10-29 21:30:19 -07:00
Steve Yegge
a7f86341a7 bd sync: 2025-10-29 21:21:13 2025-10-29 21:21:15 -07:00
Steve Yegge
c9704f8bd4 Fix export: clear stale export_hashes 2025-10-29 21:19:29 -07:00
Steve Yegge
98b153f2a7 bd sync: apply rename detection from import 2025-10-29 21:16:13 -07:00
Steve Yegge
a001e4ee7b bd sync: 2025-10-29 21:14:31 2025-10-29 21:14:31 -07:00
Steve Yegge
c9e6757f9e bd sync: 2025-10-29 20:48:48 2025-10-29 20:48:48 -07:00
Steve Yegge
4a9ee2f1e7 bd sync: 2025-10-29 20:48:00 2025-10-29 20:48:00 -07:00
Steve Yegge
4d62cb944c bd sync: 2025-10-29 20:45:07 2025-10-29 20:45:07 -07:00
Steve Yegge
5766bcb496 Import remapped issues after collision 2025-10-29 20:02:51 -07:00
Steve Yegge
5b752903ae bd sync: 2025-10-29 20:02:48 2025-10-29 20:02:48 -07:00
Steve Yegge
08bdac1e14 bd daemon sync: 2025-10-29 19:52:19 2025-10-29 19:52:19 -07:00
Steve Yegge
6e6e7eb1f5 bd daemon sync: 2025-10-29 19:52:15 2025-10-29 19:52:15 -07:00
Steve Yegge
ba18638da2 bd sync: 2025-10-29 19:51:50 2025-10-29 19:51:50 -07:00
Steve Yegge
71cc695a7d Add issue bd-102: --json flag consistency 2025-10-29 19:49:26 -07:00
Steve Yegge
e972295c1e Sync beads database 2025-10-29 19:45:50 -07:00
Steve Yegge
17a20748cb Mark in-progress issues as open 2025-10-29 19:44:54 -07:00
Steve Yegge
0f3b826010 Resolve merge conflict: use local JSONL 2025-10-29 19:32:03 -07:00
Steve Yegge
98c39c2e6f bd sync: 2025-10-29 19:31:28 2025-10-29 19:31:28 -07:00
Steve Yegge
3fd3760a28 bd daemon export: 2025-10-29 19:12:54 2025-10-29 19:12:54 -07:00
Steve Yegge
1873764370 bd daemon export: 2025-10-29 19:11:58 2025-10-29 19:11:58 -07:00
Steve Yegge
a9a26275d5 bd daemon export: 2025-10-29 19:11:28 2025-10-29 19:11:28 -07:00
Steve Yegge
6045e87d55 bd daemon export: 2025-10-29 19:08:03 2025-10-29 19:08:03 -07:00
Steve Yegge
cb68ca3781 bd daemon sync: 2025-10-29 19:06:11 2025-10-29 19:06:11 -07:00
Steve Yegge
8e85ececd2 bd sync: 2025-10-29 17:49:41 2025-10-29 17:49:41 -07:00
Steve Yegge
5743b48ca2 bd sync: 2025-10-29 16:43:32 2025-10-29 16:43:32 -07:00
Steve Yegge
29326d84c3 bd sync: 2025-10-29 16:15:13 2025-10-29 16:15:13 -07:00
Steve Yegge
1238a0a689 bd sync: 2025-10-29 15:53:34 2025-10-29 15:53:34 -07:00
Steve Yegge
8a7c36fe70 Validation and testing improvements (bd-74, bd-77)
- Remove unreachable function DroppedEventsCount from RPC server
- Add TestMutationToExportLatency for event-driven daemon validation
- Test currently skipped pending full bd-85 implementation
- Create test coverage improvement issues (bd-114 through bd-118)
- All validation checks pass: tests, build, linting baseline

Completed: bd-74, bd-77
Amp-Thread-ID: https://ampcode.com/threads/T-24404401-6c5b-466d-9045-0da3a70cff9a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 15:30:47 -07:00
Steve Yegge
da9773c31b Add bd repair-deps command (bd-58)
- Scans all issues for dependencies pointing to non-existent issues
- Reports orphaned dependencies with issue ID, depends_on ID, and type
- --fix flag automatically removes all orphaned dependencies
- --interactive mode prompts for each orphan before removal
- Uses direct SQL deletion to avoid foreign key errors on missing issues
- JSON output support with --json flag
- 4 comprehensive tests covering scan, fix, and multiple dependency types

Amp-Thread-ID: https://ampcode.com/threads/T-942a3e75-f90b-45b4-9f88-c7f1b8298cef
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 12:56:51 -07:00
Steve Yegge
2f11eaa54b bd sync: 2025-10-29 11:50:20 2025-10-29 11:50:20 -07:00
Steve Yegge
7d0cccdccb Add comprehensive unit tests for FileWatcher (bd-78)
- Test JSONL change detection with fsnotify
- Test multiple changes debounced into single action
- Test git ref change detection (platform-aware, skips if unsupported)
- Test file removal/recreation handling (platform-aware)
- Test polling fallback mode
- Test polling detects file disappearance
- Test proper cleanup with Close()

All 7 tests pass. Two tests skip gracefully on platforms where
fsnotify doesn't support git ref watching or file removal/recreation
events. No linter warnings for new test file.

Closes bd-78

Amp-Thread-ID: https://ampcode.com/threads/T-76e7b2ba-150c-461f-83e2-4a6d509d6b53
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 11:47:40 -07:00
Steve Yegge
df71bca0a1 bd sync: 2025-10-29 11:26:50 2025-10-29 11:26:50 -07:00
Steve Yegge
fea86f9b31 Implement event-driven daemon improvements for bd-85
- Add mutation events for label/dep/comment operations
- Create separate export-only and import-only functions
- Add dropped events counter with safety net export
- Complete bd-80 mutation channel implementation

Event-driven mode now:
- Emits mutation events for ALL write operations (not just create/update/close)
- Uses createExportFunc() for mutations (export+commit/push only, no pull)
- Uses createAutoImportFunc() for file changes (pull+import only, no export)
- Tracks dropped events and triggers safety export every 60s if any dropped
- Achieves <500ms latency target by avoiding full sync on each trigger

Behind BEADS_DAEMON_MODE=events flag (poll is still default)
2025-10-29 11:22:29 -07:00
Steve Yegge
757bdf6f7e bd sync: 2025-10-29 11:08:21 2025-10-29 11:08:21 -07:00
Steve Yegge
ebb425388c bd-109: Add retry logic and race condition handling for N-way collisions
- Added ExecInTransaction helper for atomic database operations
- Added IsUniqueConstraintError to detect UNIQUE constraint violations
- Wrapped RemapCollisions with retry logic (3 attempts with counter sync)
- Enhanced handleRename to detect race conditions where target ID exists
- Added defensive checks for when old ID has been deleted by another clone

Progress: Improves N-way collision handling, though full solution requires
more work (tracked in bd-108). Tests now reach later convergence rounds
before hitting complex collision scenarios.

Amp-Thread-ID: https://ampcode.com/threads/T-2b850a80-f8bd-4e38-b661-e33d1cfa7281
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 10:45:25 -07:00
Steve Yegge
7ed8d49652 bd sync: 2025-10-29 10:43:46 2025-10-29 10:43:46 -07:00
Steve Yegge
5498ec5a6d Add issues for N-way collision resolution improvements
Created four issues to address N-way collision limitations:

- bd-109: Add transaction + retry logic (P1, immediate fix)
- bd-110: Implement clone-scoped ID allocation (P2, medium-term)
- bd-111: Investigate jujutsu VCS for better merging (P2, research)
- bd-112: CRDT-based architecture for v2.0 (P3, long-term)

These address the KNOWN LIMITATION documented in beads_nway_test.go
where 5+ clone collisions fail during convergence with UNIQUE constraint
violations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:29:38 -07:00
Steve Yegge
2469559558 Add comprehensive N-way collision tests (bd-107)
- Created beads_nway_test.go with generalized N-clone collision testing
- Implemented TestFiveCloneCollision with 3 sync order variations
- Added TestTenCloneCollision for scaling verification
- Added TestEdgeCases (identical content, one different, mixed collisions)
- Added TestConvergenceTime for bounded convergence verification
- Tests document known limitation: UNIQUE constraint failures during
  convergence when multiple clones remap to same target ID
- Pattern matches TestThreeCloneCollision approach of documenting
  current behavior for future improvement

Amp-Thread-ID: https://ampcode.com/threads/T-012146d5-1841-4187-8619-230063e7711d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 22:15:45 -07:00
Steve Yegge
6662b6f057 bd sync: 2025-10-28 21:22:26 2025-10-28 21:22:26 -07:00
Steve Yegge
ff53ce26a4 Add comprehensive N-way collision tests for bd-99 2025-10-28 20:47:40 -07:00
Steve Yegge
9d0d6648bd Close bd-99 2025-10-28 20:47:34 -07:00
Steve Yegge
187e90bb0d bd sync: 2025-10-28 20:47:28 2025-10-28 20:47:28 -07:00