Commit Graph

245 Commits

Author SHA1 Message Date
Steve Yegge
1849ffe17a test: add comprehensive Comments API tests (bd-d68f)
- Test AddIssueComment basic functionality
- Test GetIssueComments retrieval and ordering
- Test edge cases (empty, nonexistent issues)
- Test dirty flag marking
- Test comment isolation across issues

Improves sqlite package coverage: 69.1% → 70.6%
2025-11-01 22:53:38 -07:00
Steve Yegge
d86f359e63 fix: DetectCycles SQL bug and add comprehensive tests
- Fix SQL query bug preventing cycle detection (bd-9f20)
  - Allow revisiting start node to complete cycle
  - Remove duplicate start_id concatenation in final SELECT
- Add cycle_detection_test.go with comprehensive test coverage (bd-cdf7)
  - Simple 2-node cycles
  - Complex multi-node cycles (4-node, 10-node)
  - Self-loops
  - Multiple independent cycles
  - Acyclic graphs (diamond, chain)
  - Empty graph and single node edge cases
  - Mixed dependency types

Improves sqlite package coverage: 68.2% → 69.1%
2025-11-01 22:51:58 -07:00
Steve Yegge
e9bb1ac121 test: add sqlite collision and delete test coverage
- Add collision_test.go: comprehensive ID collision handling tests
- Add delete_test.go: test soft delete, cascade, and edge cases
- Improves sqlite package coverage from 57.8% to 68.2%

Amp-Thread-ID: https://ampcode.com/threads/T-f4a96cad-a2a4-4f6f-af47-cd56700429f7
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 22:47:07 -07:00
Steve Yegge
6e8907335f Add test coverage for daemon lifecycle and config modules
- Add config_test.go: tests for daemon configuration (local/global, sync behavior)
- Add daemon_test.go: tests for daemon lifecycle (creation, shutdown, database path resolution)
- Add process_test.go: tests for daemon lock acquisition and release
- Closes bd-2b34.6, bd-2b34.7

Amp-Thread-ID: https://ampcode.com/threads/T-4419d1ab-4105-4e75-bea8-1837ee80e2c2
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 21:27:09 -07:00
Steve Yegge
3fc8922c86 Refactor daemon runtime into internal/daemonrunner package
- Extract Daemon struct with Start/Stop methods
- Create focused modules: config, daemon, rpc, sync, git, process, logger
- Move platform-specific code to separate files (signals_*.go, flock_*.go)
- Add database fingerprint validation
- Keep import logic in cmd/bd for now (TODO: extract later)
- All builds pass, most daemon tests pass

Closes bd-b5a3
2025-11-01 20:02:30 -07:00
Nikolai Prokoschenko
c65cfa1ebd Add dependency and dependent counts to bd list JSON output (#198)
When using `bd list --json`, each issue now includes:
- `dependency_count`: Number of issues this issue depends on
- `dependent_count`: Number of issues that depend on this issue

This provides quick access to dependency relationship counts without
needing to fetch full dependency lists or run multiple bd show commands.

Performance:
- Uses single bulk query (GetDependencyCounts) instead of N individual queries
- Overhead: ~26% for 500 issues (24ms vs 19ms baseline)
- Avoids N+1 query problem that would have caused 2.2x slowdown

Implementation:
- Added GetDependencyCounts() to Storage interface for bulk counting
- Efficient SQLite query using UNION ALL + GROUP BY
- Memory storage implementation for testing
- Moved IssueWithCounts to types package to avoid duplication
- Both RPC and direct modes use optimized bulk query

Tests:
- Added comprehensive tests for GetDependencyCounts
- Tests cover: normal operation, empty list, nonexistent IDs
- All existing tests continue to pass

Backwards compatible: JSON structure is additive, all original fields preserved.

Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-01 19:59:15 -07:00
Midworld Kim
21ab565819 Fix hyphenated issue prefix detection (#199)
- update prefix/number parsing to use the last hyphen across utils and nodb paths
- add regression tests covering multi-hyphen prefixes in both packages
2025-11-01 19:57:37 -07:00
Steve Yegge
b5db80c412 Refactor sqlite.go: Extract hash IDs, batch ops, validators (bd-90a5, bd-c796, bd-d9e0)
- Extract hash ID generation to hash_ids.go (bd-90a5)
  - generateHashID, getNextChildNumber, GetNextChildID
  - Reduced sqlite.go from 1880 to 1799 lines

- Extract batch operations to batch_ops.go (bd-c796)
  - validateBatchIssues, generateBatchIDs, bulkInsertIssues
  - bulkRecordEvents, bulkMarkDirty, CreateIssues
  - Reduced sqlite.go from 1799 to 1511 lines

- Extract validation functions to validators.go (bd-d9e0)
  - validatePriority, validateStatus, validateIssueType
  - validateTitle, validateEstimatedMinutes, validateFieldUpdate
  - Reduced sqlite.go from 1511 to 1447 lines

- Add comprehensive validator tests (bd-3b7f)
  - validators_test.go with full coverage

Total reduction: 2298 → 1447 lines (851 lines extracted, 37% reduction)

Part of epic bd-fc2d to modularize sqlite.go
All tests pass

Amp-Thread-ID: https://ampcode.com/threads/T-09c4383b-bc5c-455e-be24-02b4f9df7d78
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 19:55:48 -07:00
Steve Yegge
b5839b656d Fix compilation errors in internal/daemonrunner package
Created missing files:
- logger.go: Logger type, setupLogger method, and env helpers
- signals_unix.go: Unix signal definitions (SIGTERM, SIGINT, SIGHUP)
- signals_windows.go: Windows signal definitions
- sync.go: Sync loop implementation with export/import/validation helpers

Fixed errors:
- Added missing 'version' parameter to acquireDaemonLock call
- Removed duplicate setupLock method from process.go (kept in daemon.go)
- Removed duplicate startRPCServer from daemon.go (kept in rpc.go)
- Fixed LogPath -> LogFile config field reference
- Removed unused 'io' import from process.go

Implementation notes:
- exportToJSONL: Full implementation with dependencies, labels, comments
- importFromJSONL: Placeholder (TODO: extract from cmd/bd/import.go)
- countDBIssues: Uses SQL COUNT(*) optimization with fallback
- validatePostImport: Checks for data loss
- runSyncLoop/runEventLoop: Main daemon event loops with signal handling

All packages now compile successfully with 'go build ./...'

Amp-Thread-ID: https://ampcode.com/threads/T-36a7f730-3420-426f-9e23-f13d5fa089c4
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 19:10:27 -07:00
Steve Yegge
94f2dc1198 WIP: Extract daemon runtime into internal/daemonrunner
- Created config.go with Config struct
- Created daemon.go with Daemon struct and Start/Stop methods
- Created logger.go for logging setup
- Created process.go for lock/PID management
- Created fingerprint.go for database validation
- Created flock_unix.go/flock_windows.go for platform-specific locking
- Created git.go for git operations

Still TODO:
- Implement runGlobalDaemon, startRPCServer, runSyncLoop
- Create sync.go, rpc.go, jsonl.go, validation.go
- Update cmd/bd/daemon.go to use daemonrunner

Part of bd-5f26
2025-11-01 11:47:41 -07:00
Steve Yegge
a4b4778ced test: improve coverage for importer and sqlite utils
- Fix TestImportIssues_Update by adding timestamps to test issue
- Add comprehensive tests for sqlite utility functions (IsUniqueConstraintError, QueryContext, BeginTx, ExecInTransaction)
- Coverage improvements: sqlite util.go 0% -> 95%, sqlite package 56.4% -> 57.1%

Amp-Thread-ID: https://ampcode.com/threads/T-17e6a3e4-f881-4f53-b670-bdd796d58f68
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 11:35:31 -07:00
Steve Yegge
cc7918daf4 Implement bd stale command (bd-c01f, closes #184)
- Add bd stale command to find abandoned/forgotten issues
- Support --days (default 30), --status, --limit, --json flags
- Implement GetStaleIssues in SQLite and Memory storage
- Add full RPC/daemon support
- Comprehensive test suite (6 tests, all passing)
- Update AGENTS.md documentation

Resolves GitHub issue #184

Amp-Thread-ID: https://ampcode.com/threads/T-f021ddb8-54e3-41bf-ba7a-071749663c1d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 23:03:56 -07:00
Steve Yegge
079effdaeb Fix bd-373c: Surface daemon errors when multiple .db files exist
When daemon detects multiple .db files (after filtering .backup and vc.db),
it now writes detailed error to .beads/daemon-error file before exiting.

The error file is checked and displayed when:
- Daemon discovery fails to connect
- Auto-start fails to yield a running daemon
- User runs 'bd daemons list'

This makes the error immediately visible without requiring users to check
daemon logs.

Changes:
- cmd/bd/daemon.go: Write daemon-error file on multiple .db detection
- internal/daemon/discovery.go: Read and surface daemon-error in DaemonInfo.Error
- cmd/bd/main.go: Display daemon-error when auto-start fails

Amp-Thread-ID: https://ampcode.com/threads/T-1005a8d1-7a5a-4844-ad2d-2b8a6145825f
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 22:01:45 -07:00
Steve Yegge
0b852f52d9 Remove obsolete renumber and stale commands, fix test
- Remove renumber.go: Hash IDs eliminated need for ID compaction
- Remove stale.go: Executor/heartbeat tables were never implemented
- Fix TestListCommand: duplicate dependency constraint violation
- Update comments removing references to removed commands

Amp-Thread-ID: https://ampcode.com/threads/T-3dcd8681-c7d3-4fe1-9750-b38279b56cdb
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 21:27:33 -07:00
Steve Yegge
e2bb4311f1 Improve mutation channel robustness
- Add event type constants (MutationCreate, MutationUpdate, MutationDelete, MutationComment)
- Make buffer size configurable via BEADS_MUTATION_BUFFER (default 512, up from 100)
- Add defensive closed-channel handling in event loop consumer
- Add 1s dropped-events ticker (down from 60s) for faster recovery
- Verify mutationChan is never closed (prevents panic)

Oracle review findings addressed:
- Eliminates panic risk from send-on-closed-channel
- Reduces worst-case recovery latency from 60s to 1s
- Increases buffer capacity for better burst handling
- Type-safe event constants prevent string typos

Related: bd-36320a04, bd-1f4086c5
2025-10-31 19:12:02 -07:00
Steve Yegge
253caef761 Fix bd-e55c: Import respects updated_at timestamps
- Import now checks timestamps before updating issues
- Only applies updates if incoming version is newer than local
- Prevents older remote versions from overwriting newer local changes
- Added comprehensive tests for timestamp precedence
- Fixes issue where git pull would revert local changes to open status
2025-10-31 18:06:05 -07:00
Steve Yegge
f1dd1e4446 test: improve internal/importer coverage from 33.5% to 70.5%
- Add comprehensive tests for ImportIssues main function
- Test basic import, updates, dry-run, dependencies, labels
- Test helper functions: getOrCreateStore, GetPrefixList
- All tests passing
2025-10-31 18:01:43 -07:00
Steve Yegge
54391798a0 Add tests for internal/importer field comparison functions
- Added tests for equalPtrStr (0% -> 100% coverage)
- Added tests for equalIssueType (50% -> 100% coverage)
- Improved internal/importer coverage from 31.2% to 33.5%
- Overall project coverage increased to 46.0%
2025-10-31 17:54:14 -07:00
Steve Yegge
cd8cf997c9 Add comprehensive tests for internal/utils ID parsing functions
- Added tests for ExtractIssuePrefix (100% coverage)
- Added tests for ExtractIssueNumber (100% coverage)
- Enhanced ResolvePartialID tests with edge cases
- Added ResolvePartialIDs error handling tests
- Improved internal/utils coverage from 42% to 88%
- Overall project coverage increased from 45.6% to 45.8%

Amp-Thread-ID: https://ampcode.com/threads/T-1a44c987-7e49-431d-8e0b-467157d079eb
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 17:52:36 -07:00
Steve Yegge
caf0161ed1 Add unit tests for nodb.go and daemon/discovery.go
- Added tests for extractIssuePrefix, loadIssuesFromJSONL, detectPrefix, writeIssuesToJSONL
- Added tests for walkWithDepth depth limiting and hidden directory skipping
- Added tests for DiscoverDaemons registry and legacy discovery paths
- Improved test coverage for cmd/bd and internal/daemon
2025-10-31 17:26:37 -07:00
Steve Yegge
8bf6b1eb63 Add unit tests for autoimport, importer, and main CLI
Amp-Thread-ID: https://ampcode.com/threads/T-b89cad6b-636f-477f-925d-4c3e3f769215
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 17:17:33 -07:00
Steve Yegge
39d5608497 Merge branch 'main' of github.com:steveyegge/beads
# Conflicts:
#	.beads/beads.jsonl
2025-10-31 15:12:08 -07:00
Steve Yegge
ddab26315f Remove dead code found by deadcode analyzer
- Removed 5 unreachable functions (~200 LOC)
- computeIssueContentHash, shouldSkipExport from autoflush.go
- addDependencyUnchecked, removeDependencyIfExists from dependencies.go
- isUniqueConstraintError alias from util.go
- All tests still pass
- Closes bd-7c5915ae
2025-10-31 15:12:01 -07:00
Steve Yegge
aa567f6b9a feat: add Mermaid.js format for dependency tree visualization (#191)
Adds `bd dep tree --format mermaid` to export dependency trees as Mermaid.js flowcharts.

Features:
- Status indicators: ☐ open, ◧ in_progress, ⚠ blocked, ☑ closed
- Theme-agnostic design
- Works with --reverse flag
- Comprehensive unit tests following TDD

Co-authored-by: David Laing <david@davidlaing.com>
2025-10-31 15:11:29 -07:00
Steve Yegge
d5488cb97f Remove collision-era language from docs and code
- Updated FAQ.md, ADVANCED.md, TROUBLESHOOTING.md to explain hash IDs eliminate collisions
- Removed --resolve-collisions references from all documentation and examples
- Renamed handleCollisions() to detectUpdates() to reflect update semantics
- Updated test names: TestAutoImportWithCollision → TestAutoImportWithUpdate
- Clarified: with hash IDs, same-ID = update operation, not collision

Closes: bd-50a7, bd-b84f, bd-bda8, bd-650c, bd-3ef2, bd-c083, bd-85a6
2025-10-31 14:24:50 -07:00
Steve Yegge
eb18d342f4 Fix: Treat same-ID updates as normal updates, not collisions (bd-0134cc5a) 2025-10-31 13:39:38 -07:00
Steve Yegge
9225114c0b Fix tests after --resolve-collisions removal
Remove obsolete test file and update remaining tests to not use the
removed --resolve-collisions flag. Hash-based IDs make collision
resolution unnecessary.

Changes:
- Delete internal/rpc/server_autoimport_test.go (obsolete)
- Remove --resolve-collisions from beads_nway_test.go
- Remove --resolve-collisions from beads_twoclone_test.go
- Remove --resolve-collisions from cmd/bd/sync.go

All tests now pass.
2025-10-31 01:25:05 -07:00
Steve Yegge
0725c33fcc Remove vestigial --resolve-collisions flag
Hash-based IDs make collision resolution unnecessary. The flag was
already non-functional (handleCollisions returns error on collision
regardless of flag value).

Removed:
- --resolve-collisions flag from bd import
- ResolveCollisions field from ImportOptions and importer.Options
- All references in daemon, auto-import, and tests
- Updated error messages to reflect hash IDs don't collide

All import tests pass.

Amp-Thread-ID: https://ampcode.com/threads/T-47dfa0cc-bb71-4467-ac86-f0966a7c5d58
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 01:07:42 -07:00
Steve Yegge
64fe51d6bb Remove obsolete collision remapping code and tests
- Deleted collision remapping tests (obsolete with hash IDs bd-8e05)
- Simplified collision.go from 704 to 138 lines
- Removed RemapCollisions, ScoreCollisions, and reference update code
- Removed issue_counters table dependencies (bd-807b)
- Added COLLISION_MATH.md documentation
- Fixed RenameCounterPrefix and ResetCounter to be no-ops
- Closed bd-a58f, bd-3d65, bd-807b

Hash-based IDs make collision remapping unnecessary since collisions
are extremely rare (same ID = same content).

Amp-Thread-ID: https://ampcode.com/threads/T-cbb0f111-6a95-4598-b03e-c137112f9875
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 00:19:42 -07:00
Steve Yegge
5d137ffeeb Remove sequential ID generation and SyncAllCounters (bd-c7af, bd-8e05, bd-4c74)
- Removed SyncAllCounters() and all call sites (already no-op with hash IDs)
- Removed AllocateNextID() and getNextIDForPrefix() - sequential ID generation
- Removed collision remapping logic in internal/storage/sqlite/collision.go
- Removed rename collision handling in internal/importer/importer.go
- Removed branch-merge example (collision resolution no longer needed)
- Updated EXTENDING.md to remove counter sync examples

These were all deprecated code paths for sequential IDs that are obsolete
with hash-based IDs. Hash ID collisions are handled by extending the hash,
not by remapping to new sequential IDs.
2025-10-30 22:24:42 -07:00
Steve Yegge
e3afecca37 Remove sequential ID code path (bd-aa744b)
- Removed nextSequentialID() and getIDMode() functions
- Removed issue_counters table from schema
- Made SyncAllCounters() a no-op for backward compatibility
- Simplified ID generation to hash-only (adaptive length)
- Removed id_mode config setting
- Removed sequential ID tests and migration code
- Updated CONFIG.md and AGENTS.md to remove sequential ID references

Follow-up bd-2a70 will remove obsolete test files and renumber command.
2025-10-30 21:51:39 -07:00
Steve Yegge
76d3403d0a Implement adaptive ID length scaling (bd-ea2a13)
- Start with 4-char IDs for small databases (0-500 issues)
- Scale to 5-char at 500-1500 issues, 6-char at 1500+
- Configurable via max_collision_prob, min/max_hash_length
- Birthday paradox math ensures collision probability stays under threshold
- Comprehensive tests and documentation
- Collision calculator tool for analysis

Also filed bd-aa744b to remove sequential ID code path.
2025-10-30 21:40:52 -07:00
Steve Yegge
11c26d5af8 Add substring ID matching for all bd commands
- Enhanced ResolvePartialID to handle:
  - Bare hashes: 07b8c8 → bd-07b8c8
  - Prefix without hyphen: bd07b8c8 → bd-07b8c8
  - Full IDs: bd-07b8c8 (unchanged)
  - Substring matching: 07b → finds bd-07b8c8

- Added RPC support:
  - New OpResolveID operation
  - handleResolveID server handler
  - ResolveID client method

- Updated all commands to resolve IDs:
  - show, update, close, reopen
  - dep (add, remove, tree)
  - label (add, remove, list)

- Works in both daemon and direct modes

Fixes bd-0591c3
2025-10-30 19:20:50 -07:00
Steve Yegge
dacf8de993 Implement daemon registry system (bd-07b8c8)
- Created global daemon registry at ~/.beads/registry.json
- Daemons auto-register on start, unregister on graceful shutdown
- DiscoverDaemons() now uses registry instead of filesystem scan
- Instant daemon discovery (35ms vs indefinite hang)
- Auto-cleanup of stale registry entries
- Full test coverage

Closes bd-07b8c8, bd-acb971c7
2025-10-30 18:37:12 -07:00
Steve Yegge
cd7bdb301d Generate 6-char hash IDs with progressive 7/8-char fallback on collision (bd-7c87cf24)
- Changed generateHashID to start with 6 chars (3 bytes), expand to 7/8 on collision
- Updated both CreateIssue and CreateIssues (batch) to use progressive length fallback
- Updated tests to accept 9-11 char IDs (bd- + 6-8 hex chars)
- All new issues now generate with shorter, more readable IDs
- Existing 8-char IDs preserved (no migration needed)

Amp-Thread-ID: https://ampcode.com/threads/T-8a6058af-9f42-4bff-be02-8c8bce41eeb5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-30 18:16:24 -07:00
Steve Yegge
1fbfe58ba7 Make hash IDs opt-in via id_mode config (bd-5404)
- Add id_mode config (sequential|hash), defaults to sequential
- Update CreateIssue/CreateIssues to check id_mode and generate appropriate IDs
- Implement lazy counter initialization from existing issues
- Update migrate --to-hash-ids to set id_mode=hash after migration
- Fix hash ID tests to set id_mode=hash
- Fix renumber test to use explicit IDs
- All 183 test packages pass

This makes hash IDs backward-compatible opt-in rather than forced default.
2025-10-30 16:50:38 -07:00
Steve Yegge
2eb4c883ab Implement prefix-optional ID parsing (bd-170)
- Add internal/utils/id_parser.go with ParseIssueID and ResolvePartialID
- Update all CLI commands to accept IDs without prefix (e.g., '170' or 'bd-170')
- Add comprehensive tests for ID parsing functionality
- Works in direct mode; RPC handlers to be updated in bd-177

Commands updated:
- show, update, edit, close (show.go)
- reopen (reopen.go)
- dep add/remove/tree (dep.go)
- label add/remove/list (label.go)
- comments (comments.go)

Amp-Thread-ID: https://ampcode.com/threads/T-1f6a301b-b53f-440f-bd79-e453234ac1c9
Co-authored-by: Amp <amp@ampcode.com>
2025-10-30 15:41:46 -07:00
Steve Yegge
3ed2aa07cb Implement hierarchical child ID generation (bd-171)
- Add GetNextChildID to storage interface for generating child IDs
- Implement in SQLiteStorage with atomic counter using child_counters table
- Implement in MemoryStorage with in-memory counter
- Add --parent flag to bd create command
- Support hierarchical IDs (bd-a3f8e9.1, bd-a3f8e9.1.5) in CreateIssue
- Validate parent exists when creating hierarchical issues
- Enforce max depth of 3 levels
- Update ID validation to accept hierarchical IDs with dots
- Add comprehensive tests for child ID generation
- Manual testing confirms: sequential children, nested hierarchies, depth enforcement
2025-10-30 14:42:08 -07:00
Steve Yegge
2276d5e428 Implement hash ID generation (bd-168)
- Add generateHashID function with SHA256-based IDs
- Update CreateIssue and CreateIssues to use hash IDs
- Add collision detection with nonce retry logic
- Add comprehensive tests for hash ID generation
- Hash IDs format: prefix-<8 hex chars> (e.g., bd-a3f8e9a2)

Amp-Thread-ID: https://ampcode.com/threads/T-48f75379-427f-4d72-bbc2-42bad0d0d62d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-30 14:16:35 -07:00
Steve Yegge
6091e87cd1 Revert "Implement hash ID generation (bd-168)"
This reverts commit 2480316248.
2025-10-30 14:16:24 -07:00
Steve Yegge
2480316248 Implement hash ID generation (bd-168)
- Add generateHashID function with SHA256-based IDs
- Update CreateIssue and CreateIssues to use hash IDs
- Add collision detection with nonce retry logic
- Add comprehensive tests for hash ID generation
- Hash IDs format: prefix-<8 hex chars> (e.g., bd-a3f8e9a2)

Amp-Thread-ID: https://ampcode.com/threads/T-48f75379-427f-4d72-bbc2-42bad0d0d62d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-30 14:12:29 -07:00
Steve Yegge
2b05ec65f8 Implement 6-char progressive hash IDs (bd-166, bd-167)
- Hash ID generation now returns full 64-char SHA256
- Progressive collision handling: 6→7→8 chars on INSERT failure
- Added child_counters table for hierarchical IDs
- Updated all docs to reflect 6-char design
- Collision math: 97% of 1K issues stay at 6 chars

Next: Implement progressive retry logic in CreateIssue (bd-168)
Amp-Thread-ID: https://ampcode.com/threads/T-9931c1b7-c989-47a1-8e6a-a04469bd937d
Co-authored-by: Amp <amp@ampcode.com>
2025-10-30 14:04:03 -07:00
Steve Yegge
c34b93fa1a Fix bd-160: Implement JSONL integrity validation and prevent export deduplication data loss
## Problem
Export deduplication feature broke when JSONL and export_hashes diverged
(e.g., after git pull/reset). This caused exports to skip issues that
weren't actually in the file, leading to silent data loss.

## Solution
1. JSONL integrity validation before every export
   - Store JSONL file hash after export
   - Validate hash before export, clear export_hashes if mismatch
   - Automatically recovers from git operations changing JSONL

2. Clear export_hashes on all imports
   - Prevents stale hashes from causing future export failures
   - Import operations invalidate export_hashes state

3. Add Storage interface methods:
   - GetJSONLFileHash/SetJSONLFileHash for integrity tracking
   - ClearAllExportHashes for recovery

## Tests Added
- TestJSONLIntegrityValidation: Unit tests for validation logic
- TestImportClearsExportHashes: Verifies imports clear hashes
- TestExportIntegrityAfterJSONLTruncation: Simulates git reset (would have caught bd-160)
- TestExportIntegrityAfterJSONLDeletion: Tests recovery from file deletion
- TestMultipleExportsStayConsistent: Tests repeated export integrity

## Follow-up
Created bd-179 epic for remaining integration test gaps (multi-repo sync,
daemon auto-sync, corruption recovery tests).

Closes bd-160
2025-10-29 21:57:15 -07:00
Steve Yegge
84fd068e3f Add TestSymmetricCollision unit test for deterministic collision resolution 2025-10-29 17:49:46 -07:00
Steve Yegge
8a7c36fe70 Validation and testing improvements (bd-74, bd-77)
- Remove unreachable function DroppedEventsCount from RPC server
- Add TestMutationToExportLatency for event-driven daemon validation
- Test currently skipped pending full bd-85 implementation
- Create test coverage improvement issues (bd-114 through bd-118)
- All validation checks pass: tests, build, linting baseline

Completed: bd-74, bd-77
Amp-Thread-ID: https://ampcode.com/threads/T-24404401-6c5b-466d-9045-0da3a70cff9a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 15:30:47 -07:00
Steve Yegge
fea86f9b31 Implement event-driven daemon improvements for bd-85
- Add mutation events for label/dep/comment operations
- Create separate export-only and import-only functions
- Add dropped events counter with safety net export
- Complete bd-80 mutation channel implementation

Event-driven mode now:
- Emits mutation events for ALL write operations (not just create/update/close)
- Uses createExportFunc() for mutations (export+commit/push only, no pull)
- Uses createAutoImportFunc() for file changes (pull+import only, no export)
- Tracks dropped events and triggers safety export every 60s if any dropped
- Achieves <500ms latency target by avoiding full sync on each trigger

Behind BEADS_DAEMON_MODE=events flag (poll is still default)
2025-10-29 11:22:29 -07:00
Steve Yegge
55f803a7c9 Fix multi-round convergence for N-way collisions (bd-108)
- Add AllocateNextID() public method to SQLiteStorage for cross-package ID allocation
- Enhance handleRename() to handle collision during rename with retry logic
- Fix stale ID map issue by removing deleted IDs from dbByID after rename
- Update edge case tests to use convergence rounds consistently
- All N-way collision tests now pass (TestFiveCloneCollision, TestEdgeCases)
2025-10-29 11:08:28 -07:00
Steve Yegge
ebb425388c bd-109: Add retry logic and race condition handling for N-way collisions
- Added ExecInTransaction helper for atomic database operations
- Added IsUniqueConstraintError to detect UNIQUE constraint violations
- Wrapped RemapCollisions with retry logic (3 attempts with counter sync)
- Enhanced handleRename to detect race conditions where target ID exists
- Added defensive checks for when old ID has been deleted by another clone

Progress: Improves N-way collision handling, though full solution requires
more work (tracked in bd-108). Tests now reach later convergence rounds
before hitting complex collision scenarios.

Amp-Thread-ID: https://ampcode.com/threads/T-2b850a80-f8bd-4e38-b661-e33d1cfa7281
Co-authored-by: Amp <amp@ampcode.com>
2025-10-29 10:45:25 -07:00
Steve Yegge
ff02615f61 Implement content-first idempotent import (bd-98)
- Refactored upsertIssues to match by content hash first, then by ID
- Added buildHashMap, buildIDMap, and handleRename helper functions
- Import now detects and handles renames (same content, different ID)
- Importing same data multiple times is idempotent (reports Unchanged)
- Exported BuildReplacementCache and ReplaceIDReferencesWithCache for reuse
- All 30+ existing import tests pass
- Improved convergence for N-way collision scenarios

Changes:
- internal/importer/importer.go: Content-first matching in upsertIssues
- internal/storage/sqlite/collision.go: Exported helper functions
- internal/storage/sqlite/collision_test.go: Updated function names

Amp-Thread-ID: https://ampcode.com/threads/T-3df96ad8-7c0e-4190-87b5-6d5327718f0a
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 20:40:36 -07:00
Steve Yegge
b0d28bbdbd Remove spurious collision-related code after ultrathink review
After 2 weeks of collision/stale-data fixes, reviewed all changes to identify
spurious code that is no longer needed after content-hash resolution was implemented.

**Removed:**
1. countReferences() function from collision.go (lines 274-328)
   - Was used for reference-count based collision scoring
   - Completely unused after switching to content-hash based resolution (commit 2e87329)
   - Still exists in duplicates.go for deduplication (different use case)

2. ReferenceScore field from CollisionDetail struct
   - Marked as DEPRECATED but never removed
   - No longer used by ScoreCollisions() which now uses content hashing

3. TestCountReferences and TestCountReferencesWordBoundary tests
   - Tested the now-deleted countReferences() function
   - No longer relevant

**Fixed:**
- Updated CheckpointWAL comments to remove misleading "staleness detection" claim
  - Staleness detection uses metadata (last_import_time), NOT file mtime
  - CheckpointWAL is still valuable for data persistence and WAL size reduction
  - Comments now accurately reflect actual benefits

**Verified:**
- All tests pass (internal/storage/sqlite)
- Content-hash collision resolution still works correctly
- No behavioral changes, just cleanup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 20:07:01 -07:00