* feat(sync): implement pull-first synchronization strategy
- Add --pull-first flag and logic to sync command
- Introduce 3-way merge stub for issue synchronization
- Add concurrent edit tests for the pull-first flow
Ensures local changes are reconciled with remote updates before pushing to prevent data loss.
* feat(sync): implement 3-way merge and state tracking
- Implement 3-way merge algorithm for issue synchronization
- Add base state storage to track changes between syncs
- Add comprehensive tests for merge logic and persistence
Ensures data consistency and prevents data loss during concurrent
issue updates.
* feat(sync): implement field-level conflict merging
- Implement field-level merge logic for issue conflicts
- Add unit tests for field-level merge strategies
Reduces manual intervention by automatically resolving overlapping updates at the field level.
* refactor(sync): simplify sync flow by removing ZFC checks
The previous sync implementation relied on Zero-False-Convergence (ZFC)
staleness checks which are redundant following the transition to
structural 3-way merging. This legacy logic added complexity and
maintenance overhead without providing additional safety.
This commit introduces a streamlined sync pipeline:
- Remove ZFC staleness validation from primary sync flow
- Update safety documentation to reflect current merge strategy
- Eliminate deprecated unit tests associated with ZFC logic
These changes reduce codebase complexity while maintaining data
integrity through the robust structural 3-way merge implementation.
* feat(sync): default to pull-first sync workflow
- Set pull-first as the primary synchronization workflow
- Refactor core sync logic for better maintainability
- Update concurrent edit tests to validate 3-way merge logic
Reduces merge conflicts by ensuring local state is current before pushing changes.
* refactor(sync): clean up lint issues in merge code
- Remove unused error return from MergeIssues (never returned error)
- Use _ prefix for unused _base parameter in mergeFieldLevel
- Update callers to not expect error from MergeIssues
- Keep nolint:gosec for trusted internal file path
* test(sync): add mode compatibility and upgrade safety tests
Add tests addressing Steve's PR #918 review concerns:
- TestSyncBranchModeWithPullFirst: Verifies sync-branch config
storage and git branch creation work with pull-first
- TestExternalBeadsDirWithPullFirst: Verifies external BEADS_DIR
detection and pullFromExternalBeadsRepo
- TestUpgradeFromOldSync: Validates upgrade safety when
sync_base.jsonl doesn't exist (first sync after upgrade)
- TestMergeIssuesWithBaseState: Comprehensive 3-way merge cases
- TestLabelUnionMerge: Verifies labels use union (no data loss)
Key upgrade behavior validated:
- base=nil (no sync_base.jsonl) safely handles all cases
- Local-only issues kept (StrategyLocal)
- Remote-only issues kept (StrategyRemote)
- Overlapping issues merged (LWW scalars, union labels)
* fix(sync): report line numbers for malformed JSON
Problem:
- JSON decoding errors when loading sync base state lacked line numbers
- Difficult to identify location of syntax errors in large state files
Solution:
- Include line number reporting in JSON decoder errors during state loading
- Add regression tests for malformed sync base file scenarios
Impact:
- Users receive actionable feedback for corrupted state files
- Faster troubleshooting of manual configuration errors
* fix(sync): warn on large clock skew during sync
Problem:
- Unsynchronized clocks between systems could lead to silent merge errors
- No mechanism existed to alert users of significant timestamp drift
Solution:
- Implement clock skew detection during sync merge
- Log a warning when large timestamp differences are found
- Add comprehensive unit tests for skew reporting
Impact:
- Users are alerted to potential synchronization risks
- Easier debugging of time-related merge issues
* fix(sync): defer state update until remote push succeeds
Problem:
- Base state updated before confirming remote push completion
- Failed pushes resulted in inconsistent local state tracking
Solution:
- Defer base state update until after the remote push succeeds
Impact:
- Ensures local state accurately reflects remote repository status
- Prevents state desynchronization during network or push failures
* fix(sync): prevent concurrent sync operations
Problem:
- Multiple sync processes could run simultaneously
- Overlapping operations risk data corruption and race conditions
Solution:
- Implement file-based locking using gofrs/flock
- Add integration tests to verify locking behavior
Impact:
- Guarantees execution of a single sync process at a time
- Eliminates potential for data inconsistency during sync
* docs: document sync architecture and merge model
- Detail the 3-way merge model logic
- Describe the core synchronization architecture principles
* fix(lint): explicitly ignore lock.Unlock return value
errcheck linter flagged bare defer lock.Unlock() calls. Wrap in
anonymous function with explicit _ assignment to acknowledge
intentional ignore of unlock errors during cleanup.
* fix(lint): add sync_merge.go to G304 exclusions
The loadBaseState and saveBaseState functions use file paths derived
from trusted internal sources (beadsDir parameter from config). Add
to existing G304 exclusion list for safe JSONL file operations.
* feat(sync): integrate sync-branch into pull-first flow
When sync.branch is configured, doPullFirstSync now:
- Calls PullFromSyncBranch before merge
- Calls CommitToSyncBranch after export
This ensures sync-branch mode uses the correct branch for
pull/push operations.
* test(sync): add E2E tests for sync-branch and external BEADS_DIR
Adds comprehensive end-to-end tests:
- TestSyncBranchE2E: verifies pull→merge→commit flow with remote changes
- TestExternalBeadsDirE2E: verifies sync with separate beads repository
- TestExternalBeadsDirDetection: edge cases for repo detection
- TestCommitToExternalBeadsRepo: commit handling
* refactor(sync): remove unused rollbackJSONLFromGit
Function was defined but never called. Pull-first flow saves base
state after successful push, making this safety net unnecessary.
* test(sync): add export-only mode E2E test
Add TestExportOnlySync to cover --no-pull flag which was the only
untested sync mode. This completes full mode coverage:
- Normal (pull-first): sync_test.go, sync_merge_test.go
- Sync-branch: sync_modes_test.go:TestSyncBranchE2E (PR#918)
- External BEADS_DIR: sync_external_test.go (PR#918)
- From-main: sync_branch_priority_test.go
- Local-only: sync_local_only_test.go
- Export-only: sync_modes_test.go:TestExportOnlySync (this commit)
Refs: #911
* docs(sync): add sync modes reference section
Document all 6 sync modes with triggers, flows, and use cases.
Include mode selection decision tree and test coverage matrix.
Co-authored-by: Claude <noreply@anthropic.com>
* test(sync): upgrade sync-branch E2E tests to bare repo
- Replace mocked repository with real bare repo setup
- Implement multi-machine simulation in sync tests
- Refactor test logic to handle distributed states
Coverage: sync-branch end-to-end scenarios
* test(sync): add daemon sync-branch E2E tests
- Implement E2E tests for daemon sync-branch flow
- Add test cases for force-overwrite scenarios
Coverage: daemon sync-branch workflow in cmd/bd
* docs(sync): document sync-branch paths and E2E architecture
- Describe sync-branch CLI and Daemon execution flow
- Document the end-to-end test architecture
* build(nix): update vendorHash for gofrs/flock dependency
New dependency added for file-based sync locking changes the
Go module checksum.
---------
Co-authored-by: Claude <noreply@anthropic.com>
The root cause was isExternalBeadsDir() incorrectly identifying bare repo
worktrees as "external" repos. This caused bd sync to take the "external
beads dir" code path instead of the worktree-based sync branch path.
The bug: isExternalBeadsDir() compared syncbranch.GetRepoRoot() (which returns
incorrect values for bare repo worktrees) with getRepoRootFromPath() (which
uses --show-toplevel). These returned different values for bare repo
worktrees, causing local worktrees to be treated as external.
The fix: Use git rev-parse --git-common-dir for comparison instead of repo
root. This correctly identifies that worktrees of the same repo share the
same git directory, regardless of bare repo setup.
Also added:
- getGitCommonDir() helper function
- Tests for both getGitCommonDir and isExternalBeadsDir
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was previously skipped because it called importFromJSONL which spawns
a subprocess. During tests, os.Executable() returns the test binary rather
than the bd binary, causing the subprocess to fail.
Changes:
- Remove t.Skip() and TODO comment
- Parse JSONL directly using encoding/json instead of subprocess
- Call importIssuesCore directly instead of importFromJSONL
- Update DB count expectation to reflect actual import behavior (additive)
The core ZFC fix behavior is preserved: when divergence is detected,
import runs and export is skipped to protect the JSONL source of truth.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Created 7 new beads to track TODOs that were previously inline comments:
- bd-7l27: Integrate migration detection into bd doctor (5 files)
- bd-6x6g: Add multi-repo target repo switching in bd create
- bd-ag35: Add daemon RPC endpoints for config and stale check
- bd-2dwo: Remove deprecated daemon logger function
- bd-0qx5: Implement Jira issue timestamp comparison for sync
- bd-7zka: Implement formula features (repeat, for_each, branch, gate, split)
- bd-h048: Refactor sync_test to use direct import logic
Updated 16 TODO comments to include bead ID references for tracking.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The sync sanitize process was incorrectly removing newly created issues
when they happened to have IDs matching entries in the deletions manifest.
This could occur with hash-based IDs when content is similar to previously
deleted issues.
The fix adds protection for issues that were in the left snapshot (local
export before pull). These represent local work and should not be removed
by sanitize, even if they match entries in the deletions manifest.
Changes:
- Load left snapshot in sanitizeJSONLWithDeletions() to build protection set
- Add protection check before removing issues from JSONL
- Add ProtectedCount/ProtectedIDs to SanitizeResult for tracking
- Log protected issues during sync for visibility
- Add comprehensive test coverage for the fix
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The sanitizeJSONLWithDeletions function was incorrectly removing ALL issues
whose ID appeared in deletions.jsonl, including tombstones. This caused:
1. Second sync after delete: tombstone removed from JSONL by sanitize
2. Import sees ID in deletions.jsonl but no tombstone in JSONL
3. Import creates new tombstone via convertDeletionToTombstone
4. UNIQUE constraint error: tombstone already exists in DB
The fix checks the issue status and only removes non-tombstone issues.
Tombstones are the proper representation of deletions and must be preserved.
Added test: TestSanitizeJSONLWithDeletions_PreservesTombstones
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replaces manual working directory save/restore patterns
with Go's built-in `t.Chdir()` helper across 23 test files.
The manual pattern involved calling `os.Getwd()` to save
the original directory, using `defer os.Chdir(origWd)` for
restoration, and manually handling errors during directory
changes. This boilerplate has been replaced with single
`t.Chdir(path)` calls that handle cleanup automatically.
The `t.Chdir()` method automatically restores the working
directory when the test completes, eliminating the need for
manual defer statements and error handling.
Total:
~75 instances replaced (assuming Claude's math is right)
Co-authored-by: Claude <noreply@anthropic.com>
- Increase Windows test timeout from 20m to 30m
- Add -parallel=4 flag to allow concurrent test execution
- Add t.Parallel() to safe table-driven tests in validate_test.go,
autoimport_test.go, and sync_test.go
This should prevent the Windows CI timeout caused by the cumulative
runtime of cmd/bd tests exceeding 20 minutes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes#417: When using --from-main mode (either explicitly or auto-detected),
git history backfill now defaults to disabled. This prevents creating
incorrect deletion records for locally-created beads that don't exist in
main's git history.
Changes:
- Add resolveNoGitHistoryForFromMain() helper function
- Apply noGitHistory=true for both explicit and auto-detected from-main mode
- Add comprehensive unit tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: GraemeF <graeme@graemef.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Root cause: When beads.db is deleted and recreated while daemon is running,
daemon's SQLite connection becomes stale (points to old deleted file via
file descriptor), causing export to return incomplete/corrupt data.
Fix:
- sync command now forces direct mode by closing daemonClient at start
- importFromJSONL subprocess uses --no-daemon to avoid daemon connection issues
- Added documentation to import.go explaining the daemon behavior
Also:
- Skip TestZFCSkipsExportAfterImport (broken test - subprocess spawning
doesn't work in test environment, needs refactoring
- Update hook templates to version 0.26.2
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
EOF
)
The existing ZFC checks only compared issue counts, missing the case where
counts match but content differs (e.g., status=open vs status=closed).
Added Case 3 (bd-f2f) hash-based staleness detection:
- Before export, check if JSONL content hash differs from stored hash
- If hash mismatch detected, import JSONL first to get remote changes
- Then proceed with export to write merged state
This prevents the corruption scenario where:
1. Stale DB has old status values (e.g., status=closed)
2. Remote JSONL has correct values (e.g., status=open)
3. Export would overwrite correct JSONL with stale DB values
4. Git 3-way merge would propagate the corruption
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Short tests were timing out after 13+ minutes due to:
1. TestZFCSkipsExportAfterImport spawning subprocess that tried to
auto-start daemon - now skipped in short mode
2. TestVersionFlag taking 5+ seconds because --version flag did not
skip PersistentPreRun daemon startup - added early return
3. TestGetVersionsSince* had hardcoded version expectations that
became stale - made tests dynamic using versionChanges array
Short tests now complete in ~8 seconds instead of timing out.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add JSONL sanitization after git pull to remove deleted issues that
git's 3-way merge may resurrect. Also add bd doctor check to hydrate
deletions.jsonl from git history for pre-v0.25.0 deletions.
Changes:
- Add sanitizeJSONLWithDeletions() in sync.go (Step 3.6)
- Add checkDeletionsManifest() in doctor.go (Check 18)
- Add HydrateDeletionsManifest() fix in doctor/fix/deletions.go
- Add looksLikeIssueID() validation to prevent false positives
- Add comprehensive tests for sanitization logic
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
- Add Count function to deletions package for fast line counting
- Add maybeAutoCompactDeletions to sync (opt-in via deletions.auto_compact config)
- Fix regex escaping in batchCheckGitHistory (bd-bgs)
- Add 30s timeout to git history commands (bd-f0n)
- Use git rev-parse --show-toplevel for proper repo root detection (bd-bhd)
- Add tests for Count and auto-compact functionality
Closes: bd-qsm, bd-bgs, bd-f0n, bd-bhd
## Problem
When bd sync detected stale DB (>50% divergence), it would import JSONL to fix the DB,
but then immediately export the DB back to JSONL. This caused the stale DB to overwrite
the JSONL after a git pull, undoing cleanup work.
Example scenario:
1. Clone has 688 stale issues in DB (628 closed)
2. git pull brings JSONL with 62 issues (cleanup applied)
3. bd sync detects 1009.7% divergence and imports JSONL (DB → 62 issues) ✓
4. bd sync exports DB to JSONL (JSONL still 62 issues) ✓
5. But this marks JSONL as "changed" and commits/pushes it ✗
## Solution
After ZFC (JSONL First Consistency) import, set skipExport flag to prevent the export step.
JSONL is the source of truth after import - DB should sync to match, not export back.
## Changes
- cmd/bd/sync.go: Add skipExport flag, set it after ZFC import
- cmd/bd/sync.go: Wrap export logic in `if !skipExport` block
- CHANGELOG.md: Update ZFC entry with accurate description
- cmd/bd/sync_test.go: Add TestZFCSkipsExportAfterImport
Fixes #bd-l0r
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add setupGitRepo(), setupGitRepoWithBranch(), and setupMinimalGitRepo() helpers
- Refactor 19 test functions to use shared git repo setup
- Reduces duplicate git initialization boilerplate by ~300 lines
- All tests pass with improved maintainability
Related to bd-ktng
- Add isInRebase() to detect rebase state
- Add hasJSONLConflict() to check for JSONL-only conflicts
- Add runGitRebaseContinue() to continue rebase after resolution
- Auto-export from DB and resolve conflict when detected
- Add comprehensive tests for auto-resolution logic
Implements bd-cwmt
Fixes daemon and bd sync to honor BEADS_SYNC_BRANCH environment variable
as documented in PROTECTED_BRANCHES.md for CI/CD temporary overrides.
Changes:
- Updated internal/syncbranch.Get() to prioritize env var over DB config
- Both daemon sync and bd sync CLI now use syncbranch.Get()
- Added comprehensive tests for env var override behavior
- Validates branch names using git-style rules
This enables CI/CD workflows to override sync branch per-job without
mutating database config.
Based on PR #364 by Charles P. Cross <cpdata@users.noreply.github.com>
Co-authored-by: Charles P. Cross <cpdata@users.noreply.github.com>
- Adds checkGitHooks() function to verify recommended hooks are installed
- Checks for pre-commit, post-merge, and pre-push hooks
- Warns if hooks are missing with install instructions
- Shows up early in diagnostics (even if .beads/ missing)
- Includes comprehensive test coverage
- Filed bd-6049 for broken --json flag
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
