Commit Graph

846 Commits

Author SHA1 Message Date
Steve Yegge
ff53ce26a4 Add comprehensive N-way collision tests for bd-99 2025-10-28 20:47:40 -07:00
Steve Yegge
9d0d6648bd Close bd-99 2025-10-28 20:47:34 -07:00
Steve Yegge
187e90bb0d bd sync: 2025-10-28 20:47:28 2025-10-28 20:47:28 -07:00
Steve Yegge
ff02615f61 Implement content-first idempotent import (bd-98)
- Refactored upsertIssues to match by content hash first, then by ID
- Added buildHashMap, buildIDMap, and handleRename helper functions
- Import now detects and handles renames (same content, different ID)
- Importing same data multiple times is idempotent (reports Unchanged)
- Exported BuildReplacementCache and ReplaceIDReferencesWithCache for reuse
- All 30+ existing import tests pass
- Improved convergence for N-way collision scenarios

Changes:
- internal/importer/importer.go: Content-first matching in upsertIssues
- internal/storage/sqlite/collision.go: Exported helper functions
- internal/storage/sqlite/collision_test.go: Updated function names

Amp-Thread-ID: https://ampcode.com/threads/T-3df96ad8-7c0e-4190-87b5-6d5327718f0a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 20:40:36 -07:00
Steve Yegge
44df03ae55 bd sync: 2025-10-28 20:21:42 2025-10-28 20:21:42 -07:00
Steve Yegge
9b2d3554c3 Implement global N-way collision resolution (bd-97)
- Add ResolveNWayCollisions function for deterministic global resolution
- Replace pairwise ScoreCollisions approach with content-hash-based sorting
- Implement helper functions: groupCollisionsByID, deduplicateVersionsByContentHash, extractNumber
- Update importer.go to use new resolution flow
- Add applyNWayResolution to create remapped issues
- Export UpdateReferences for use by importer
- Add comprehensive tests for N-way collision resolution
- All tests pass including TestTwoCloneCollision

This enables proper convergence for N-way (3+) collision scenarios where
multiple clones create issues with the same ID but different content.
The algorithm groups all versions, deduplicates by content, sorts by hash,
and assigns sequential IDs deterministically.

Amp-Thread-ID: https://ampcode.com/threads/T-3e649e13-792c-4fb5-a342-58fa805eb517
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 20:07:51 -07:00
Steve Yegge
b0d28bbdbd Remove spurious collision-related code after ultrathink review
After 2 weeks of collision/stale-data fixes, reviewed all changes to identify
spurious code that is no longer needed after content-hash resolution was implemented.

**Removed:**
1. countReferences() function from collision.go (lines 274-328)
   - Was used for reference-count based collision scoring
   - Completely unused after switching to content-hash based resolution (commit 2e87329)
   - Still exists in duplicates.go for deduplication (different use case)

2. ReferenceScore field from CollisionDetail struct
   - Marked as DEPRECATED but never removed
   - No longer used by ScoreCollisions() which now uses content hashing

3. TestCountReferences and TestCountReferencesWordBoundary tests
   - Tested the now-deleted countReferences() function
   - No longer relevant

**Fixed:**
- Updated CheckpointWAL comments to remove misleading "staleness detection" claim
  - Staleness detection uses metadata (last_import_time), NOT file mtime
  - CheckpointWAL is still valuable for data persistence and WAL size reduction
  - Comments now accurately reflect actual benefits

**Verified:**
- All tests pass (internal/storage/sqlite)
- Content-hash collision resolution still works correctly
- No behavioral changes, just cleanup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 20:07:01 -07:00
Steve Yegge
a51a9297d6 bd sync: 2025-10-28 19:54:04 2025-10-28 19:54:04 -07:00
Steve Yegge
a40b27a03d Add repair commands: bd repair-deps, bd detect-pollution, bd validate
Implements Phase 1 of bd-56 (Repair Commands & AI-Assisted Tooling):

New commands:
- bd repair-deps: Find and fix orphaned dependency references
- bd detect-pollution: Detect test issues using pattern matching
- bd validate: Comprehensive health check (orphans, duplicates, pollution)

Features:
- JSON output support for all commands
- Safe deletion with backup for detect-pollution
- Auto-fix support for orphaned dependencies
- Direct storage access (requires BEADS_NO_DAEMON=1)

Closes bd-56 (Phase 1 complete)
Related: bd-103, bd-105, bd-106

Amp-Thread-ID: https://ampcode.com/threads/T-5822c6d2-d645-4043-9a8d-3c51ac93bbb7
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 19:46:12 -07:00
Steve Yegge
dd6af39906 bd sync: 2025-10-28 19:38:15 2025-10-28 19:38:17 -07:00
Steve Yegge
6ac42499bf removed obsolete/implemented designs 2025-10-28 19:31:58 -07:00
Steve Yegge
176dc0d601 Fix bd-66: Add robust polling fallback for file watcher
- Detect fsnotify.NewWatcher() errors and auto-fallback to polling mode
- Add BEADS_WATCHER_FALLBACK env var to control behavior (default: enabled)
- Poll every 5 seconds with comprehensive change detection:
  - Track file existence, size, and mtime to catch all changes
  - Handle file disappearance/reappearance correctly
  - Trigger on file recreation even with older timestamps
- Fix goroutine leak: Close() now stops background goroutines via cancel context
- Tighten git refs filtering to only trigger for events under .git/refs/heads
- Trigger after successful JSONL rewatch on rename/remove events
- Improve logging to show actual poll interval in warnings

All tests passing.

Amp-Thread-ID: https://ampcode.com/threads/T-8f5edc23-4b78-4b80-b8f3-66050f45eb61
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 19:27:56 -07:00
Steve Yegge
d04821a95c bd sync: 2025-10-28 19:21:28 2025-10-28 19:21:28 -07:00
Steve Yegge
dd9541f300 Fix autoimport tests for content-hash collision scoring (bd-100) 2025-10-28 19:19:06 -07:00
Steve Yegge
9644d61de2 Make DetectCollisions read-only (bd-96)
- Add RenameDetail type to track content matches with different IDs
- Remove deletion logic from DetectCollisions (now read-only)
- Create ApplyCollisionResolution to handle all modifications
- Update importer.go to use two-phase approach (detect then apply)
- Fix dependency preservation in RemapCollisions
  - Collect all dependencies before CASCADE DELETE
  - Recreate with updated IDs after remapping
- Add tests: TestDetectCollisionsReadOnly, TestApplyCollisionResolution
- Update collision tests for content-hash scoring behavior
- Create bd-100 to track fixing autoimport tests
2025-10-28 19:16:51 -07:00
Steve Yegge
f7963945c3 bd sync: 2025-10-28 19:12:59 2025-10-28 19:12:59 -07:00
Steve Yegge
50fd617984 bd sync: 2025-10-28 19:08:17 2025-10-28 19:08:17 -07:00
Steve Yegge
d9eb273e15 Complete bd-95: Add content-addressable identity (ContentHash field) 2025-10-28 18:57:16 -07:00
Steve Yegge
ad267b5de6 bd sync: 2025-10-28 18:57:12 2025-10-28 18:57:12 -07:00
Steve Yegge
ae30c4a639 Add initial commit to bare repo in TestThreeCloneCollision 2025-10-28 18:40:27 -07:00
Steve Yegge
63f6db64d9 bd sync: 2025-10-28 18:40:03 2025-10-28 18:40:03 -07:00
Steve Yegge
f384fb239e Document N-way collision convergence problem for investigation 2025-10-28 18:24:00 -07:00
Steve Yegge
aab5be6fb1 Add TestThreeCloneCollision for 3-way collision testing (bd-92) 2025-10-28 18:09:21 -07:00
Steve Yegge
d685da41e9 bd sync: 2025-10-28 18:09:16 2025-10-28 18:09:16 -07:00
Steve Yegge
bbea17ae51 Fix TestTwoCloneCollision to compare content not timestamps (bd-91)
The test was comparing full JSON output including timestamps, causing
false negative failures. The 2-clone collision resolution actually works
correctly - both clones converge to identical semantic content.

Changes:
- Added compareIssuesIgnoringTimestamps helper to compare issue content
- Compares ID, title, description, status, priority, type (not timestamps)
- Added filterTrackedChanges to ignore untracked files in git status
- Test now passes, proving collision resolution works for 2-clone case

Closes bd-91

Amp-Thread-ID: https://ampcode.com/threads/T-a58284d2-f39f-482a-9db0-bec31c19e9e5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 18:03:17 -07:00
Steve Yegge
a687b395a4 Update bd-86 with progress on rename detection 2025-10-28 17:20:44 -07:00
Steve Yegge
ceb1b922a9 bd sync: 2025-10-28 17:19:28 2025-10-28 17:19:28 -07:00
Steve Yegge
2e87329cf8 WIP: Implement content-hash based collision resolution (bd-89)
- Replace timestamp-based collision scoring with deterministic content hashing
- Add hashIssueContent() using SHA-256 of all substantive fields
- Modify ScoreCollisions to compare hashes and set RemapIncoming flag
- Update RemapCollisions to handle both directions (remap incoming OR existing)
- Add CollisionDetail.RemapIncoming field to control which version gets remapped
- Add unit tests for hash function and deterministic collision resolution

Status: Hash-based resolution works correctly, but TestTwoCloneCollision still
fails due to missing rename detection. After Clone B resolves collision,
Clone A needs to recognize its issue was remapped to a different ID.

Next: Add content-based rename detection during import to prevent
re-resolving already-resolved collisions.

Progress on bd-86.

Amp-Thread-ID: https://ampcode.com/threads/T-b19b49e8-b52a-463d-b052-8a526a500260
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 17:11:40 -07:00
Steve Yegge
c5e7ad8d71 Add bd-86: Epic for making two-clone workflow work 2025-10-28 16:57:04 -07:00
Steve Yegge
b19c24ac55 bd sync: 2025-10-28 16:30:54 2025-10-28 16:30:54 -07:00
Steve Yegge
ef1ca3265d Keep local version (84 issues, removed duplicates) 2025-10-28 16:29:09 -07:00
Steve Yegge
923c48d5f6 bd sync: 2025-10-28 16:28:58 2025-10-28 16:28:58 -07:00
Steve Yegge
2432f30821 bd sync: 2025-10-28 16:27:07 2025-10-28 16:27:07 -07:00
Steve Yegge
d5ab6c8fa0 bd sync: merge duplicates 2025-10-28 16:26:12 -07:00
Steve Yegge
7d575a0267 bd sync: 2025-10-28 16:23:40 2025-10-28 16:23:40 -07:00
Steve Yegge
304a16c157 Merge main - keep local version
Amp-Thread-ID: https://ampcode.com/threads/T-f3675ea3-98d6-448d-a18a-9c8dc4a0c7a0
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 16:22:09 -07:00
Steve Yegge
9af5f6b997 bd sync: 2025-10-28 16:21:41 2025-10-28 16:21:41 -07:00
Steve Yegge
c45588c39c bd sync: 2025-10-28 16:20:59 2025-10-28 16:20:59 -07:00
Steve Yegge
92efd45460 bd sync: 2025-10-28 16:18:45 2025-10-28 16:18:45 -07:00
Steve Yegge
9a109902b4 Consolidate documentation: move maintainer docs to docs/, remove redundant files
- Move RELEASING.md and LINTING.md to docs/ (maintainer-only content)
- Delete WORKFLOW.md (agent workflow content belongs in AGENTS.md)
- Delete TEXT_FORMATS.md (technical details belong in ADVANCED.md)
- Update all cross-references to point to new locations
- Keep CLAUDE.md (required by Claude Code)

Reduces root-level docs from 20 to 16 files with clearer organization.

Amp-Thread-ID: https://ampcode.com/threads/T-fe1db4f3-16c6-4a79-8887-c7f4c1f11c43
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 15:46:12 -07:00
Steve Yegge
3841e6c0cf bd sync: 2025-10-28 14:48:33 2025-10-28 14:48:33 -07:00
Steve Yegge
7d366461a5 bd sync: 2025-10-28 14:22:11 2025-10-28 14:22:11 -07:00
Steve Yegge
7985214e25 bd sync: 2025-10-28 14:21:23 2025-10-28 14:21:25 -07:00
Steve Yegge
cc79e232f5 Remove unreachable functions from import_shared.go (bd-18)
- Removed renameImportedIssuePrefixes and supporting functions
- Removed unused imports (fmt, sort, strings, utils)
- Saved ~130 LOC of dead code
- All tests pass
2025-10-28 14:20:04 -07:00
Steve Yegge
ee4bd834f0 bd sync: 2025-10-28 14:19:39 2025-10-28 14:20:04 -07:00
Steve Yegge
781e300d33 Add event-driven daemon architecture (Phase 1 foundation)
- Add fsnotify dependency for file watching
- Create daemon_debouncer.go: batch rapid events (500ms window)
- Create daemon_watcher.go: monitor JSONL and git refs changes
- Create daemon_event_loop.go: event-driven sync loop
- Add mutation channel to RPC server (create/update/close events)
- Add BEADS_DAEMON_MODE env var (poll/events, default: poll)

Phase 1 implementation: opt-in via BEADS_DAEMON_MODE=events
Target: <500ms latency (vs 5000ms), ~60% CPU reduction

Related: bd-49 (epic), bd-50, bd-51, bd-53, bd-54, bd-55, bd-56
Amp-Thread-ID: https://ampcode.com/threads/T-35a3d0d7-4e19-421d-8392-63755035036e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 13:12:37 -07:00
Steve Yegge
97c676077d bd sync: 2025-10-28 12:31:47 2025-10-28 12:31:47 -07:00
Steve Yegge
96fb7ed29c bd sync: 2025-10-28 12:03:57 2025-10-28 12:03:57 -07:00
Steve Yegge
79b952fccf Optimize TestTwoCloneCollision: 3x faster with polling instead of sleeps
- Replace fixed 2s sleeps with smart polling (50-100ms intervals)
- Reduce daemon wait timeout from 5s to 1s with early exit
- Add timeout to daemon cleanup to prevent hanging
- Test now runs in ~18s instead of timing out at 60s

Amp-Thread-ID: https://ampcode.com/threads/T-51945255-d5fd-4b42-9816-c44ff4d3bdd1
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 11:52:27 -07:00
Steve Yegge
a4eb4fe451 Add two-clone collision test proving beads multi-agent workflow failure
- Creates TestTwoCloneCollision integration test
- Sets up 2 independent clones with git hooks and daemons
- Both file issues with same ID (test-1)
- Demonstrates databases don't converge after collision resolution
- Clone A: test-1='Issue from clone A', test-2='Issue from clone B'
- Clone B: test-1='Issue from clone B', test-2='Issue from clone A'
- Git status shows dirty state in both clones
- Test proves beads fails at basic multi-agent workflow

Also adds --json flag to create, ready, and list commands for
better test integration.

Amp-Thread-ID: https://ampcode.com/threads/T-8fa0ab6c-2226-4f9b-8e11-14e1156537fc
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 11:34:27 -07:00