Commit Graph

854 Commits

Author SHA1 Message Date
Steve Yegge
55f803a7c9 Fix multi-round convergence for N-way collisions (bd-108)
- Add AllocateNextID() public method to SQLiteStorage for cross-package ID allocation
- Enhance handleRename() to handle collision during rename with retry logic
- Fix stale ID map issue by removing deleted IDs from dbByID after rename
- Update edge case tests to use convergence rounds consistently
- All N-way collision tests now pass (TestFiveCloneCollision, TestEdgeCases)
2025-10-29 11:08:28 -07:00
Steve Yegge
757bdf6f7e bd sync: 2025-10-29 11:08:21 2025-10-29 11:08:21 -07:00
Steve Yegge
ebb425388c bd-109: Add retry logic and race condition handling for N-way collisions
- Added ExecInTransaction helper for atomic database operations
- Added IsUniqueConstraintError to detect UNIQUE constraint violations
- Wrapped RemapCollisions with retry logic (3 attempts with counter sync)
- Enhanced handleRename to detect race conditions where target ID exists
- Added defensive checks for when old ID has been deleted by another clone

Progress: Improves N-way collision handling, though full solution requires
more work (tracked in bd-108). Tests now reach later convergence rounds
before hitting complex collision scenarios.

Amp-Thread-ID: https://ampcode.com/threads/T-2b850a80-f8bd-4e38-b661-e33d1cfa7281
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 10:45:25 -07:00
Steve Yegge
7ed8d49652 bd sync: 2025-10-29 10:43:46 2025-10-29 10:43:46 -07:00
Steve Yegge
5498ec5a6d Add issues for N-way collision resolution improvements
Created four issues to address N-way collision limitations:

- bd-109: Add transaction + retry logic (P1, immediate fix)
- bd-110: Implement clone-scoped ID allocation (P2, medium-term)
- bd-111: Investigate jujutsu VCS for better merging (P2, research)
- bd-112: CRDT-based architecture for v2.0 (P3, long-term)

These address the KNOWN LIMITATION documented in beads_nway_test.go
where 5+ clone collisions fail during convergence with UNIQUE constraint
violations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:29:38 -07:00
Steve Yegge
2469559558 Add comprehensive N-way collision tests (bd-107)
- Created beads_nway_test.go with generalized N-clone collision testing
- Implemented TestFiveCloneCollision with 3 sync order variations
- Added TestTenCloneCollision for scaling verification
- Added TestEdgeCases (identical content, one different, mixed collisions)
- Added TestConvergenceTime for bounded convergence verification
- Tests document known limitation: UNIQUE constraint failures during
  convergence when multiple clones remap to same target ID
- Pattern matches TestThreeCloneCollision approach of documenting
  current behavior for future improvement

Amp-Thread-ID: https://ampcode.com/threads/T-012146d5-1841-4187-8619-230063e7711d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 22:15:45 -07:00
Steve Yegge
6662b6f057 bd sync: 2025-10-28 21:22:26 2025-10-28 21:22:26 -07:00
Steve Yegge
ca8c6c5ec4 Fix N-way tests to handle known convergence limitations with t.Skip 2025-10-28 20:58:03 -07:00
Steve Yegge
ff53ce26a4 Add comprehensive N-way collision tests for bd-99 2025-10-28 20:47:40 -07:00
Steve Yegge
9d0d6648bd Close bd-99 2025-10-28 20:47:34 -07:00
Steve Yegge
187e90bb0d bd sync: 2025-10-28 20:47:28 2025-10-28 20:47:28 -07:00
Steve Yegge
ff02615f61 Implement content-first idempotent import (bd-98)
- Refactored upsertIssues to match by content hash first, then by ID
- Added buildHashMap, buildIDMap, and handleRename helper functions
- Import now detects and handles renames (same content, different ID)
- Importing same data multiple times is idempotent (reports Unchanged)
- Exported BuildReplacementCache and ReplaceIDReferencesWithCache for reuse
- All 30+ existing import tests pass
- Improved convergence for N-way collision scenarios

Changes:
- internal/importer/importer.go: Content-first matching in upsertIssues
- internal/storage/sqlite/collision.go: Exported helper functions
- internal/storage/sqlite/collision_test.go: Updated function names

Amp-Thread-ID: https://ampcode.com/threads/T-3df96ad8-7c0e-4190-87b5-6d5327718f0a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 20:40:36 -07:00
Steve Yegge
44df03ae55 bd sync: 2025-10-28 20:21:42 2025-10-28 20:21:42 -07:00
Steve Yegge
9b2d3554c3 Implement global N-way collision resolution (bd-97)
- Add ResolveNWayCollisions function for deterministic global resolution
- Replace pairwise ScoreCollisions approach with content-hash-based sorting
- Implement helper functions: groupCollisionsByID, deduplicateVersionsByContentHash, extractNumber
- Update importer.go to use new resolution flow
- Add applyNWayResolution to create remapped issues
- Export UpdateReferences for use by importer
- Add comprehensive tests for N-way collision resolution
- All tests pass including TestTwoCloneCollision

This enables proper convergence for N-way (3+) collision scenarios where
multiple clones create issues with the same ID but different content.
The algorithm groups all versions, deduplicates by content, sorts by hash,
and assigns sequential IDs deterministically.

Amp-Thread-ID: https://ampcode.com/threads/T-3e649e13-792c-4fb5-a342-58fa805eb517
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 20:07:51 -07:00
Steve Yegge
b0d28bbdbd Remove spurious collision-related code after ultrathink review
After 2 weeks of collision/stale-data fixes, reviewed all changes to identify
spurious code that is no longer needed after content-hash resolution was implemented.

**Removed:**
1. countReferences() function from collision.go (lines 274-328)
   - Was used for reference-count based collision scoring
   - Completely unused after switching to content-hash based resolution (commit 2e87329)
   - Still exists in duplicates.go for deduplication (different use case)

2. ReferenceScore field from CollisionDetail struct
   - Marked as DEPRECATED but never removed
   - No longer used by ScoreCollisions() which now uses content hashing

3. TestCountReferences and TestCountReferencesWordBoundary tests
   - Tested the now-deleted countReferences() function
   - No longer relevant

**Fixed:**
- Updated CheckpointWAL comments to remove misleading "staleness detection" claim
  - Staleness detection uses metadata (last_import_time), NOT file mtime
  - CheckpointWAL is still valuable for data persistence and WAL size reduction
  - Comments now accurately reflect actual benefits

**Verified:**
- All tests pass (internal/storage/sqlite)
- Content-hash collision resolution still works correctly
- No behavioral changes, just cleanup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 20:07:01 -07:00
Steve Yegge
a51a9297d6 bd sync: 2025-10-28 19:54:04 2025-10-28 19:54:04 -07:00
Steve Yegge
a40b27a03d Add repair commands: bd repair-deps, bd detect-pollution, bd validate
Implements Phase 1 of bd-56 (Repair Commands & AI-Assisted Tooling):

New commands:
- bd repair-deps: Find and fix orphaned dependency references
- bd detect-pollution: Detect test issues using pattern matching
- bd validate: Comprehensive health check (orphans, duplicates, pollution)

Features:
- JSON output support for all commands
- Safe deletion with backup for detect-pollution
- Auto-fix support for orphaned dependencies
- Direct storage access (requires BEADS_NO_DAEMON=1)

Closes bd-56 (Phase 1 complete)
Related: bd-103, bd-105, bd-106

Amp-Thread-ID: https://ampcode.com/threads/T-5822c6d2-d645-4043-9a8d-3c51ac93bbb7
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 19:46:12 -07:00
Steve Yegge
dd6af39906 bd sync: 2025-10-28 19:38:15 2025-10-28 19:38:17 -07:00
Steve Yegge
6ac42499bf removed obsolete/implemented designs 2025-10-28 19:31:58 -07:00
Steve Yegge
176dc0d601 Fix bd-66: Add robust polling fallback for file watcher
- Detect fsnotify.NewWatcher() errors and auto-fallback to polling mode
- Add BEADS_WATCHER_FALLBACK env var to control behavior (default: enabled)
- Poll every 5 seconds with comprehensive change detection:
  - Track file existence, size, and mtime to catch all changes
  - Handle file disappearance/reappearance correctly
  - Trigger on file recreation even with older timestamps
- Fix goroutine leak: Close() now stops background goroutines via cancel context
- Tighten git refs filtering to only trigger for events under .git/refs/heads
- Trigger after successful JSONL rewatch on rename/remove events
- Improve logging to show actual poll interval in warnings

All tests passing.

Amp-Thread-ID: https://ampcode.com/threads/T-8f5edc23-4b78-4b80-b8f3-66050f45eb61
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 19:27:56 -07:00
Steve Yegge
d04821a95c bd sync: 2025-10-28 19:21:28 2025-10-28 19:21:28 -07:00
Steve Yegge
dd9541f300 Fix autoimport tests for content-hash collision scoring (bd-100) 2025-10-28 19:19:06 -07:00
Steve Yegge
9644d61de2 Make DetectCollisions read-only (bd-96)
- Add RenameDetail type to track content matches with different IDs
- Remove deletion logic from DetectCollisions (now read-only)
- Create ApplyCollisionResolution to handle all modifications
- Update importer.go to use two-phase approach (detect then apply)
- Fix dependency preservation in RemapCollisions
  - Collect all dependencies before CASCADE DELETE
  - Recreate with updated IDs after remapping
- Add tests: TestDetectCollisionsReadOnly, TestApplyCollisionResolution
- Update collision tests for content-hash scoring behavior
- Create bd-100 to track fixing autoimport tests
2025-10-28 19:16:51 -07:00
Steve Yegge
f7963945c3 bd sync: 2025-10-28 19:12:59 2025-10-28 19:12:59 -07:00
Steve Yegge
50fd617984 bd sync: 2025-10-28 19:08:17 2025-10-28 19:08:17 -07:00
Steve Yegge
d9eb273e15 Complete bd-95: Add content-addressable identity (ContentHash field) 2025-10-28 18:57:16 -07:00
Steve Yegge
ad267b5de6 bd sync: 2025-10-28 18:57:12 2025-10-28 18:57:12 -07:00
Steve Yegge
ae30c4a639 Add initial commit to bare repo in TestThreeCloneCollision 2025-10-28 18:40:27 -07:00
Steve Yegge
63f6db64d9 bd sync: 2025-10-28 18:40:03 2025-10-28 18:40:03 -07:00
Steve Yegge
f384fb239e Document N-way collision convergence problem for investigation 2025-10-28 18:24:00 -07:00
Steve Yegge
aab5be6fb1 Add TestThreeCloneCollision for 3-way collision testing (bd-92) 2025-10-28 18:09:21 -07:00
Steve Yegge
d685da41e9 bd sync: 2025-10-28 18:09:16 2025-10-28 18:09:16 -07:00
Steve Yegge
bbea17ae51 Fix TestTwoCloneCollision to compare content not timestamps (bd-91)
The test was comparing full JSON output including timestamps, causing
false negative failures. The 2-clone collision resolution actually works
correctly - both clones converge to identical semantic content.

Changes:
- Added compareIssuesIgnoringTimestamps helper to compare issue content
- Compares ID, title, description, status, priority, type (not timestamps)
- Added filterTrackedChanges to ignore untracked files in git status
- Test now passes, proving collision resolution works for 2-clone case

Closes bd-91

Amp-Thread-ID: https://ampcode.com/threads/T-a58284d2-f39f-482a-9db0-bec31c19e9e5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 18:03:17 -07:00
Steve Yegge
a687b395a4 Update bd-86 with progress on rename detection 2025-10-28 17:20:44 -07:00
Steve Yegge
ceb1b922a9 bd sync: 2025-10-28 17:19:28 2025-10-28 17:19:28 -07:00
Steve Yegge
2e87329cf8 WIP: Implement content-hash based collision resolution (bd-89)
- Replace timestamp-based collision scoring with deterministic content hashing
- Add hashIssueContent() using SHA-256 of all substantive fields
- Modify ScoreCollisions to compare hashes and set RemapIncoming flag
- Update RemapCollisions to handle both directions (remap incoming OR existing)
- Add CollisionDetail.RemapIncoming field to control which version gets remapped
- Add unit tests for hash function and deterministic collision resolution

Status: Hash-based resolution works correctly, but TestTwoCloneCollision still
fails due to missing rename detection. After Clone B resolves collision,
Clone A needs to recognize its issue was remapped to a different ID.

Next: Add content-based rename detection during import to prevent
re-resolving already-resolved collisions.

Progress on bd-86.

Amp-Thread-ID: https://ampcode.com/threads/T-b19b49e8-b52a-463d-b052-8a526a500260
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 17:11:40 -07:00
Steve Yegge
c5e7ad8d71 Add bd-86: Epic for making two-clone workflow work 2025-10-28 16:57:04 -07:00
Steve Yegge
b19c24ac55 bd sync: 2025-10-28 16:30:54 2025-10-28 16:30:54 -07:00
Steve Yegge
ef1ca3265d Keep local version (84 issues, removed duplicates) 2025-10-28 16:29:09 -07:00
Steve Yegge
923c48d5f6 bd sync: 2025-10-28 16:28:58 2025-10-28 16:28:58 -07:00
Steve Yegge
2432f30821 bd sync: 2025-10-28 16:27:07 2025-10-28 16:27:07 -07:00
Steve Yegge
d5ab6c8fa0 bd sync: merge duplicates 2025-10-28 16:26:12 -07:00
Steve Yegge
7d575a0267 bd sync: 2025-10-28 16:23:40 2025-10-28 16:23:40 -07:00
Steve Yegge
304a16c157 Merge main - keep local version
Amp-Thread-ID: https://ampcode.com/threads/T-f3675ea3-98d6-448d-a18a-9c8dc4a0c7a0
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 16:22:09 -07:00
Steve Yegge
9af5f6b997 bd sync: 2025-10-28 16:21:41 2025-10-28 16:21:41 -07:00
Steve Yegge
c45588c39c bd sync: 2025-10-28 16:20:59 2025-10-28 16:20:59 -07:00
Steve Yegge
92efd45460 bd sync: 2025-10-28 16:18:45 2025-10-28 16:18:45 -07:00
Steve Yegge
9a109902b4 Consolidate documentation: move maintainer docs to docs/, remove redundant files
- Move RELEASING.md and LINTING.md to docs/ (maintainer-only content)
- Delete WORKFLOW.md (agent workflow content belongs in AGENTS.md)
- Delete TEXT_FORMATS.md (technical details belong in ADVANCED.md)
- Update all cross-references to point to new locations
- Keep CLAUDE.md (required by Claude Code)

Reduces root-level docs from 20 to 16 files with clearer organization.

Amp-Thread-ID: https://ampcode.com/threads/T-fe1db4f3-16c6-4a79-8887-c7f4c1f11c43
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 15:46:12 -07:00
Steve Yegge
3841e6c0cf bd sync: 2025-10-28 14:48:33 2025-10-28 14:48:33 -07:00
Steve Yegge
7d366461a5 bd sync: 2025-10-28 14:22:11 2025-10-28 14:22:11 -07:00