For repos with 1000+ issues where less than 20% are dirty,
incremental export reads the existing JSONL, merges only
changed issues, and writes back - avoiding full re-export.
- Add exportToJSONLIncrementalDeferred as new default export path
- Add shouldUseIncrementalExport to check thresholds
- Add performIncrementalExport for merge-based export
- Add readJSONLToMap helper for fast JSONL parsing
- Falls back to full export when incremental is not beneficial
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The JSONL file hash mismatch warning was appearing consistently after
git commit operations, even when no actual data inconsistency existed.
Root Cause:
Two code paths export to JSONL but only one updated jsonl_file_hash:
- flushToJSONLWithState() (used by bd update/create): Updated both
jsonl_content_hash AND jsonl_file_hash
- finalizeExport() (used by bd sync --flush-only): Updated only
jsonl_content_hash, NOT jsonl_file_hash
Since validateJSONLIntegrity() checks jsonl_file_hash, any bd command
after a git commit would see a mismatch and trigger the warning.
Steps to Reproduce (before fix):
1. cd /path/to/beads-project
2. bd update -p 3 some-issue-id # Works fine, stores hash H1
3. git add .beads/issues.jsonl && git commit --amend --no-edit
# Pre-commit hook runs bd sync --flush-only
# This updates jsonl_content_hash to H2 but leaves jsonl_file_hash as H1
4. bd update -p 3 some-issue-id # WARNING appears!
# validateJSONLIntegrity() compares file (H2) with jsonl_file_hash (H1)
5. Repeat steps 3-4 indefinitely - warning always appears
The fix adds SetJSONLFileHash() call to finalizeExport(), ensuring both
export paths update the same metadata consistently.
* feat(sync): implement pull-first synchronization strategy
- Add --pull-first flag and logic to sync command
- Introduce 3-way merge stub for issue synchronization
- Add concurrent edit tests for the pull-first flow
Ensures local changes are reconciled with remote updates before pushing to prevent data loss.
* feat(sync): implement 3-way merge and state tracking
- Implement 3-way merge algorithm for issue synchronization
- Add base state storage to track changes between syncs
- Add comprehensive tests for merge logic and persistence
Ensures data consistency and prevents data loss during concurrent
issue updates.
* feat(sync): implement field-level conflict merging
- Implement field-level merge logic for issue conflicts
- Add unit tests for field-level merge strategies
Reduces manual intervention by automatically resolving overlapping updates at the field level.
* refactor(sync): simplify sync flow by removing ZFC checks
The previous sync implementation relied on Zero-False-Convergence (ZFC)
staleness checks which are redundant following the transition to
structural 3-way merging. This legacy logic added complexity and
maintenance overhead without providing additional safety.
This commit introduces a streamlined sync pipeline:
- Remove ZFC staleness validation from primary sync flow
- Update safety documentation to reflect current merge strategy
- Eliminate deprecated unit tests associated with ZFC logic
These changes reduce codebase complexity while maintaining data
integrity through the robust structural 3-way merge implementation.
* feat(sync): default to pull-first sync workflow
- Set pull-first as the primary synchronization workflow
- Refactor core sync logic for better maintainability
- Update concurrent edit tests to validate 3-way merge logic
Reduces merge conflicts by ensuring local state is current before pushing changes.
* refactor(sync): clean up lint issues in merge code
- Remove unused error return from MergeIssues (never returned error)
- Use _ prefix for unused _base parameter in mergeFieldLevel
- Update callers to not expect error from MergeIssues
- Keep nolint:gosec for trusted internal file path
* test(sync): add mode compatibility and upgrade safety tests
Add tests addressing Steve's PR #918 review concerns:
- TestSyncBranchModeWithPullFirst: Verifies sync-branch config
storage and git branch creation work with pull-first
- TestExternalBeadsDirWithPullFirst: Verifies external BEADS_DIR
detection and pullFromExternalBeadsRepo
- TestUpgradeFromOldSync: Validates upgrade safety when
sync_base.jsonl doesn't exist (first sync after upgrade)
- TestMergeIssuesWithBaseState: Comprehensive 3-way merge cases
- TestLabelUnionMerge: Verifies labels use union (no data loss)
Key upgrade behavior validated:
- base=nil (no sync_base.jsonl) safely handles all cases
- Local-only issues kept (StrategyLocal)
- Remote-only issues kept (StrategyRemote)
- Overlapping issues merged (LWW scalars, union labels)
* fix(sync): report line numbers for malformed JSON
Problem:
- JSON decoding errors when loading sync base state lacked line numbers
- Difficult to identify location of syntax errors in large state files
Solution:
- Include line number reporting in JSON decoder errors during state loading
- Add regression tests for malformed sync base file scenarios
Impact:
- Users receive actionable feedback for corrupted state files
- Faster troubleshooting of manual configuration errors
* fix(sync): warn on large clock skew during sync
Problem:
- Unsynchronized clocks between systems could lead to silent merge errors
- No mechanism existed to alert users of significant timestamp drift
Solution:
- Implement clock skew detection during sync merge
- Log a warning when large timestamp differences are found
- Add comprehensive unit tests for skew reporting
Impact:
- Users are alerted to potential synchronization risks
- Easier debugging of time-related merge issues
* fix(sync): defer state update until remote push succeeds
Problem:
- Base state updated before confirming remote push completion
- Failed pushes resulted in inconsistent local state tracking
Solution:
- Defer base state update until after the remote push succeeds
Impact:
- Ensures local state accurately reflects remote repository status
- Prevents state desynchronization during network or push failures
* fix(sync): prevent concurrent sync operations
Problem:
- Multiple sync processes could run simultaneously
- Overlapping operations risk data corruption and race conditions
Solution:
- Implement file-based locking using gofrs/flock
- Add integration tests to verify locking behavior
Impact:
- Guarantees execution of a single sync process at a time
- Eliminates potential for data inconsistency during sync
* docs: document sync architecture and merge model
- Detail the 3-way merge model logic
- Describe the core synchronization architecture principles
* fix(lint): explicitly ignore lock.Unlock return value
errcheck linter flagged bare defer lock.Unlock() calls. Wrap in
anonymous function with explicit _ assignment to acknowledge
intentional ignore of unlock errors during cleanup.
* fix(lint): add sync_merge.go to G304 exclusions
The loadBaseState and saveBaseState functions use file paths derived
from trusted internal sources (beadsDir parameter from config). Add
to existing G304 exclusion list for safe JSONL file operations.
* feat(sync): integrate sync-branch into pull-first flow
When sync.branch is configured, doPullFirstSync now:
- Calls PullFromSyncBranch before merge
- Calls CommitToSyncBranch after export
This ensures sync-branch mode uses the correct branch for
pull/push operations.
* test(sync): add E2E tests for sync-branch and external BEADS_DIR
Adds comprehensive end-to-end tests:
- TestSyncBranchE2E: verifies pull→merge→commit flow with remote changes
- TestExternalBeadsDirE2E: verifies sync with separate beads repository
- TestExternalBeadsDirDetection: edge cases for repo detection
- TestCommitToExternalBeadsRepo: commit handling
* refactor(sync): remove unused rollbackJSONLFromGit
Function was defined but never called. Pull-first flow saves base
state after successful push, making this safety net unnecessary.
* test(sync): add export-only mode E2E test
Add TestExportOnlySync to cover --no-pull flag which was the only
untested sync mode. This completes full mode coverage:
- Normal (pull-first): sync_test.go, sync_merge_test.go
- Sync-branch: sync_modes_test.go:TestSyncBranchE2E (PR#918)
- External BEADS_DIR: sync_external_test.go (PR#918)
- From-main: sync_branch_priority_test.go
- Local-only: sync_local_only_test.go
- Export-only: sync_modes_test.go:TestExportOnlySync (this commit)
Refs: #911
* docs(sync): add sync modes reference section
Document all 6 sync modes with triggers, flows, and use cases.
Include mode selection decision tree and test coverage matrix.
Co-authored-by: Claude <noreply@anthropic.com>
* test(sync): upgrade sync-branch E2E tests to bare repo
- Replace mocked repository with real bare repo setup
- Implement multi-machine simulation in sync tests
- Refactor test logic to handle distributed states
Coverage: sync-branch end-to-end scenarios
* test(sync): add daemon sync-branch E2E tests
- Implement E2E tests for daemon sync-branch flow
- Add test cases for force-overwrite scenarios
Coverage: daemon sync-branch workflow in cmd/bd
* docs(sync): document sync-branch paths and E2E architecture
- Describe sync-branch CLI and Daemon execution flow
- Document the end-to-end test architecture
* build(nix): update vendorHash for gofrs/flock dependency
New dependency added for file-based sync locking changes the
Go module checksum.
---------
Co-authored-by: Claude <noreply@anthropic.com>
This fixes the atomicity gap where exportToJSONL would update SQLite
metadata (clear dirty flags, update content hash, last_import_time)
BEFORE the git commit. If git commit failed, SQLite would incorrectly
indicate the sync succeeded.
Changes:
- Add ExportResult struct to capture export metadata for deferred finalization
- Add exportToJSONLDeferred() that exports without updating metadata
- Add finalizeExport() to update metadata after git commit succeeds
- Update daemon_sync.go sync flows to defer metadata updates
Now the sync flow is truly atomic: metadata only updates after git commit.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Executed-By: beads/crew/dave
Rig: beads
Role: crew
Add .beads/config.yaml support for template validation settings:
- validation.on-create: warn|error|none (default: none)
- validation.on-sync: warn|error|none (default: none)
When set to "warn", issues missing required sections (based on type) show
warnings but operations proceed. When set to "error", operations fail.
Implementation:
- Add validation keys to YamlOnlyKeys in yaml_config.go
- Add defaults in config.go
- Wire up bd create to check validation.on-create config
- Wire up bd sync to run validation before export
- Add tests for config loading
- Update CONFIG.md documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Strip (bd-xxx), (gt-xxx) suffixes from code comments and changelog
entries. The descriptions remain meaningful without the ephemeral
issue IDs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Analysis found these commands are dead code:
- gt never calls `bd pin` - uses `bd update --status=pinned` instead
- Beads.Pin() wrapper exists but is never called
- bd hook functionality duplicated by gt mol status
- Code comment says "pinned field is cosmetic for bd hook visibility"
Removed:
- cmd/bd/pin.go
- cmd/bd/unpin.go
- cmd/bd/hook.go
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The sync command's exportToJSONL function was not filtering wisps,
causing them to be re-added to JSONL during bd sync re-export step.
autoflush.go had wisp filtering but sync_export.go did not. Added
the same filtering logic to prevent wisps from being exported to JSONL.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
