6 Commits

Author SHA1 Message Date
Scott Nixon
810192157c fix(export): populate export_hashes after successful export (GH#1278) (#1286)
Child issues created with --parent were missing from export_hashes table,
which affects integrity tracking and future incremental export features.

This fix ensures SetExportHash() is called for all exported issues:
- Updated ExportResult to include IssueContentHashes map
- Updated finalizeExport() to call SetExportHash() for each exported issue
- Updated exportToJSONLDeferred() to collect content hashes during export
- Updated performIncrementalExport() to collect content hashes for dirty issues
- Updated exportToJSONLWithStore() to call SetExportHash() after export
- Updated daemon's handleExport() to call SetExportHash() after export

Added test TestExportPopulatesExportHashes to verify the fix works for
both regular and hierarchical (child) issue IDs.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 17:10:02 -08:00
Steve Yegge
2c86404d65 fix: resolve P2 sync noise and cleanup issues
- bd-6pni: Auto-filter tombstoned issues with mismatched prefixes during
  import instead of failing. Tombstones from contributor PRs with different
  test prefixes are pollution and safe to ignore.

- bd-ffr9: Stop recreating deletions.jsonl after tombstone migration.
  Added IsTombstoneMigrationComplete() check to all code paths that write
  to the legacy deletions manifest.

- bd-admx: Fix perpetual "JSONL file hash mismatch" warning. Now clears
  both export_hashes AND jsonl_file_hash when mismatch detected, so the
  warning doesn't repeat.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 00:56:06 -08:00
Steve Yegge
57253f93a3 Context propagation with graceful cancellation (bd-rtp, bd-yb8, bd-2o2)
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.

Key changes:

1. Signal-aware contexts (bd-rtp):
   - Added rootCtx/rootCancel in main.go using signal.NotifyContext()
   - Set up in PersistentPreRun, cancelled in PersistentPostRun
   - Daemon uses same pattern in runDaemonLoop()
   - Handles SIGINT/SIGTERM for graceful shutdown

2. Context propagation (bd-yb8):
   - All commands now use rootCtx instead of context.Background()
   - sqlite.New() receives context for cancellable operations
   - Database operations respect context cancellation
   - Storage layer propagates context through all queries

3. Cancellation tests (bd-2o2):
   - Added import_cancellation_test.go with comprehensive tests
   - Added export cancellation test in export_test.go
   - Tests verify database integrity after cancellation
   - All cancellation tests passing

Fixes applied during review:
   - Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
   - Fixed test context contamination (reset rootCtx in test cleanup)
   - Fixed export tests missing context setup

Impact:
   - Pressing Ctrl+C during import/export now cancels gracefully
   - No database corruption or hanging transactions
   - Clean shutdown of all operations

Tested:
   - go build ./cmd/bd ✓
   - go test ./cmd/bd -run TestImportCancellation ✓
   - go test ./cmd/bd -run TestExportCommand ✓
   - Manual Ctrl+C testing verified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 21:57:23 -05:00
Steve Yegge
5fabb5fdcc Fix bd-c6cf: Force full export when export_hashes is cleared
When validateJSONLIntegrity() clears export_hashes due to hash mismatch
or missing JSONL, the subsequent export now correctly exports ALL issues
instead of only dirty ones, preventing permanent database divergence.

Changes:
- validateJSONLIntegrity() returns (needsFullExport, error) to signal when
  export_hashes was cleared
- flushToJSONL() moved integrity check BEFORE isDirty gate so integrity
  issues trigger export even when nothing is dirty
- Missing JSONL treated as non-fatal force-full-export case
- Increased scanner buffer from 64KB to 2MB to handle large JSON lines
- Added scanner.Err() check to catch buffer overflow errors
- Updated all tests to verify needsFullExport flag

Fixes database divergence issue where clearing export_hashes didn't
trigger re-export, causing 5 issues to disappear from JSONL in fred clone.

Amp-Thread-ID: https://ampcode.com/threads/T-bf2fdcd6-7bbd-4c30-b1db-746b928c93b8
Co-authored-by: Amp <amp@ampcode.com>
2025-11-01 20:29:13 -07:00
Steve Yegge
0725c33fcc Remove vestigial --resolve-collisions flag
Hash-based IDs make collision resolution unnecessary. The flag was
already non-functional (handleCollisions returns error on collision
regardless of flag value).

Removed:
- --resolve-collisions flag from bd import
- ResolveCollisions field from ImportOptions and importer.Options
- All references in daemon, auto-import, and tests
- Updated error messages to reflect hash IDs don't collide

All import tests pass.

Amp-Thread-ID: https://ampcode.com/threads/T-47dfa0cc-bb71-4467-ac86-f0966a7c5d58
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 01:07:42 -07:00
Steve Yegge
c34b93fa1a Fix bd-160: Implement JSONL integrity validation and prevent export deduplication data loss
## Problem
Export deduplication feature broke when JSONL and export_hashes diverged
(e.g., after git pull/reset). This caused exports to skip issues that
weren't actually in the file, leading to silent data loss.

## Solution
1. JSONL integrity validation before every export
   - Store JSONL file hash after export
   - Validate hash before export, clear export_hashes if mismatch
   - Automatically recovers from git operations changing JSONL

2. Clear export_hashes on all imports
   - Prevents stale hashes from causing future export failures
   - Import operations invalidate export_hashes state

3. Add Storage interface methods:
   - GetJSONLFileHash/SetJSONLFileHash for integrity tracking
   - ClearAllExportHashes for recovery

## Tests Added
- TestJSONLIntegrityValidation: Unit tests for validation logic
- TestImportClearsExportHashes: Verifies imports clear hashes
- TestExportIntegrityAfterJSONLTruncation: Simulates git reset (would have caught bd-160)
- TestExportIntegrityAfterJSONLDeletion: Tests recovery from file deletion
- TestMultipleExportsStayConsistent: Tests repeated export integrity

## Follow-up
Created bd-179 epic for remaining integration test gaps (multi-repo sync,
daemon auto-sync, corruption recovery tests).

Closes bd-160
2025-10-29 21:57:15 -07:00