* feat(sync): implement pull-first synchronization strategy - Add --pull-first flag and logic to sync command - Introduce 3-way merge stub for issue synchronization - Add concurrent edit tests for the pull-first flow Ensures local changes are reconciled with remote updates before pushing to prevent data loss. * feat(sync): implement 3-way merge and state tracking - Implement 3-way merge algorithm for issue synchronization - Add base state storage to track changes between syncs - Add comprehensive tests for merge logic and persistence Ensures data consistency and prevents data loss during concurrent issue updates. * feat(sync): implement field-level conflict merging - Implement field-level merge logic for issue conflicts - Add unit tests for field-level merge strategies Reduces manual intervention by automatically resolving overlapping updates at the field level. * refactor(sync): simplify sync flow by removing ZFC checks The previous sync implementation relied on Zero-False-Convergence (ZFC) staleness checks which are redundant following the transition to structural 3-way merging. This legacy logic added complexity and maintenance overhead without providing additional safety. This commit introduces a streamlined sync pipeline: - Remove ZFC staleness validation from primary sync flow - Update safety documentation to reflect current merge strategy - Eliminate deprecated unit tests associated with ZFC logic These changes reduce codebase complexity while maintaining data integrity through the robust structural 3-way merge implementation. * feat(sync): default to pull-first sync workflow - Set pull-first as the primary synchronization workflow - Refactor core sync logic for better maintainability - Update concurrent edit tests to validate 3-way merge logic Reduces merge conflicts by ensuring local state is current before pushing changes. * refactor(sync): clean up lint issues in merge code - Remove unused error return from MergeIssues (never returned error) - Use _ prefix for unused _base parameter in mergeFieldLevel - Update callers to not expect error from MergeIssues - Keep nolint:gosec for trusted internal file path * test(sync): add mode compatibility and upgrade safety tests Add tests addressing Steve's PR #918 review concerns: - TestSyncBranchModeWithPullFirst: Verifies sync-branch config storage and git branch creation work with pull-first - TestExternalBeadsDirWithPullFirst: Verifies external BEADS_DIR detection and pullFromExternalBeadsRepo - TestUpgradeFromOldSync: Validates upgrade safety when sync_base.jsonl doesn't exist (first sync after upgrade) - TestMergeIssuesWithBaseState: Comprehensive 3-way merge cases - TestLabelUnionMerge: Verifies labels use union (no data loss) Key upgrade behavior validated: - base=nil (no sync_base.jsonl) safely handles all cases - Local-only issues kept (StrategyLocal) - Remote-only issues kept (StrategyRemote) - Overlapping issues merged (LWW scalars, union labels) * fix(sync): report line numbers for malformed JSON Problem: - JSON decoding errors when loading sync base state lacked line numbers - Difficult to identify location of syntax errors in large state files Solution: - Include line number reporting in JSON decoder errors during state loading - Add regression tests for malformed sync base file scenarios Impact: - Users receive actionable feedback for corrupted state files - Faster troubleshooting of manual configuration errors * fix(sync): warn on large clock skew during sync Problem: - Unsynchronized clocks between systems could lead to silent merge errors - No mechanism existed to alert users of significant timestamp drift Solution: - Implement clock skew detection during sync merge - Log a warning when large timestamp differences are found - Add comprehensive unit tests for skew reporting Impact: - Users are alerted to potential synchronization risks - Easier debugging of time-related merge issues * fix(sync): defer state update until remote push succeeds Problem: - Base state updated before confirming remote push completion - Failed pushes resulted in inconsistent local state tracking Solution: - Defer base state update until after the remote push succeeds Impact: - Ensures local state accurately reflects remote repository status - Prevents state desynchronization during network or push failures * fix(sync): prevent concurrent sync operations Problem: - Multiple sync processes could run simultaneously - Overlapping operations risk data corruption and race conditions Solution: - Implement file-based locking using gofrs/flock - Add integration tests to verify locking behavior Impact: - Guarantees execution of a single sync process at a time - Eliminates potential for data inconsistency during sync * docs: document sync architecture and merge model - Detail the 3-way merge model logic - Describe the core synchronization architecture principles * fix(lint): explicitly ignore lock.Unlock return value errcheck linter flagged bare defer lock.Unlock() calls. Wrap in anonymous function with explicit _ assignment to acknowledge intentional ignore of unlock errors during cleanup. * fix(lint): add sync_merge.go to G304 exclusions The loadBaseState and saveBaseState functions use file paths derived from trusted internal sources (beadsDir parameter from config). Add to existing G304 exclusion list for safe JSONL file operations. * feat(sync): integrate sync-branch into pull-first flow When sync.branch is configured, doPullFirstSync now: - Calls PullFromSyncBranch before merge - Calls CommitToSyncBranch after export This ensures sync-branch mode uses the correct branch for pull/push operations. * test(sync): add E2E tests for sync-branch and external BEADS_DIR Adds comprehensive end-to-end tests: - TestSyncBranchE2E: verifies pull→merge→commit flow with remote changes - TestExternalBeadsDirE2E: verifies sync with separate beads repository - TestExternalBeadsDirDetection: edge cases for repo detection - TestCommitToExternalBeadsRepo: commit handling * refactor(sync): remove unused rollbackJSONLFromGit Function was defined but never called. Pull-first flow saves base state after successful push, making this safety net unnecessary. * test(sync): add export-only mode E2E test Add TestExportOnlySync to cover --no-pull flag which was the only untested sync mode. This completes full mode coverage: - Normal (pull-first): sync_test.go, sync_merge_test.go - Sync-branch: sync_modes_test.go:TestSyncBranchE2E (PR#918) - External BEADS_DIR: sync_external_test.go (PR#918) - From-main: sync_branch_priority_test.go - Local-only: sync_local_only_test.go - Export-only: sync_modes_test.go:TestExportOnlySync (this commit) Refs: #911 * docs(sync): add sync modes reference section Document all 6 sync modes with triggers, flows, and use cases. Include mode selection decision tree and test coverage matrix. Co-authored-by: Claude <noreply@anthropic.com> * test(sync): upgrade sync-branch E2E tests to bare repo - Replace mocked repository with real bare repo setup - Implement multi-machine simulation in sync tests - Refactor test logic to handle distributed states Coverage: sync-branch end-to-end scenarios * test(sync): add daemon sync-branch E2E tests - Implement E2E tests for daemon sync-branch flow - Add test cases for force-overwrite scenarios Coverage: daemon sync-branch workflow in cmd/bd * docs(sync): document sync-branch paths and E2E architecture - Describe sync-branch CLI and Daemon execution flow - Document the end-to-end test architecture * build(nix): update vendorHash for gofrs/flock dependency New dependency added for file-based sync locking changes the Go module checksum. --------- Co-authored-by: Claude <noreply@anthropic.com>
366 lines
12 KiB
Go
366 lines
12 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/git"
|
|
)
|
|
|
|
// TestExternalBeadsDirE2E tests the full external BEADS_DIR flow.
|
|
// This is an end-to-end regression test for PR#918.
|
|
//
|
|
// When BEADS_DIR points to a separate git repository (external mode),
|
|
// sync operations should work correctly:
|
|
// 1. Changes are committed to the external beads repo (not the project repo)
|
|
// 2. Pulls from the external repo bring in remote changes
|
|
// 3. The merge algorithm works correctly across repo boundaries
|
|
func TestExternalBeadsDirE2E(t *testing.T) {
|
|
ctx := context.Background()
|
|
|
|
// Store original working directory
|
|
originalWd, err := os.Getwd()
|
|
if err != nil {
|
|
t.Fatalf("failed to get original working directory: %v", err)
|
|
}
|
|
|
|
// Setup: Create the main project repo
|
|
projectDir := t.TempDir()
|
|
if err := setupGitRepoInDir(t, projectDir); err != nil {
|
|
t.Fatalf("failed to setup project repo: %v", err)
|
|
}
|
|
|
|
// Setup: Create a separate external beads repo
|
|
// Resolve symlinks to avoid macOS /var -> /private/var issues
|
|
externalDir, err := filepath.EvalSymlinks(t.TempDir())
|
|
if err != nil {
|
|
t.Fatalf("eval symlinks failed: %v", err)
|
|
}
|
|
if err := setupGitRepoInDir(t, externalDir); err != nil {
|
|
t.Fatalf("failed to setup external repo: %v", err)
|
|
}
|
|
|
|
// Create .beads directory in external repo
|
|
externalBeadsDir := filepath.Join(externalDir, ".beads")
|
|
if err := os.MkdirAll(externalBeadsDir, 0755); err != nil {
|
|
t.Fatalf("failed to create external .beads dir: %v", err)
|
|
}
|
|
|
|
// Create issues.jsonl in external beads repo with initial issue
|
|
jsonlPath := filepath.Join(externalBeadsDir, "issues.jsonl")
|
|
issue1 := `{"id":"ext-1","title":"External Issue 1","status":"open","issue_type":"task","priority":2,"created_at":"2025-01-01T00:00:00Z","updated_at":"2025-01-01T00:00:00Z"}`
|
|
if err := os.WriteFile(jsonlPath, []byte(issue1+"\n"), 0644); err != nil {
|
|
t.Fatalf("write external JSONL failed: %v", err)
|
|
}
|
|
|
|
// Commit initial beads files in external repo
|
|
runGitInDir(t, externalDir, "add", ".beads")
|
|
runGitInDir(t, externalDir, "commit", "-m", "initial beads setup")
|
|
t.Log("✓ External beads repo initialized with issue ext-1")
|
|
|
|
// Change to project directory (simulating user's project)
|
|
if err := os.Chdir(projectDir); err != nil {
|
|
t.Fatalf("chdir to project failed: %v", err)
|
|
}
|
|
defer func() { _ = os.Chdir(originalWd) }()
|
|
|
|
// Reset git caches after directory change
|
|
git.ResetCaches()
|
|
|
|
// Test 1: isExternalBeadsDir should detect external repo
|
|
if !isExternalBeadsDir(ctx, externalBeadsDir) {
|
|
t.Error("isExternalBeadsDir should return true for external beads dir")
|
|
}
|
|
t.Log("✓ External beads dir correctly detected")
|
|
|
|
// Test 2: getRepoRootFromPath should correctly identify external repo root
|
|
repoRoot, err := getRepoRootFromPath(ctx, externalBeadsDir)
|
|
if err != nil {
|
|
t.Fatalf("getRepoRootFromPath failed: %v", err)
|
|
}
|
|
// Normalize paths for comparison
|
|
resolvedExternal, _ := filepath.EvalSymlinks(externalDir)
|
|
if repoRoot != resolvedExternal {
|
|
t.Errorf("getRepoRootFromPath = %q, want %q", repoRoot, resolvedExternal)
|
|
}
|
|
t.Logf("✓ getRepoRootFromPath correctly identifies external repo: %s", repoRoot)
|
|
|
|
// Test 3: pullFromExternalBeadsRepo should handle no-remote gracefully
|
|
err = pullFromExternalBeadsRepo(ctx, externalBeadsDir)
|
|
if err != nil {
|
|
t.Errorf("pullFromExternalBeadsRepo should handle no-remote: %v", err)
|
|
}
|
|
t.Log("✓ Pull from external beads repo handled no-remote correctly")
|
|
|
|
// Test 4: Create new issue and commit to external repo
|
|
issue2 := `{"id":"ext-2","title":"External Issue 2","status":"open","issue_type":"task","priority":2,"created_at":"2025-01-02T00:00:00Z","updated_at":"2025-01-02T00:00:00Z"}`
|
|
combinedContent := issue1 + "\n" + issue2 + "\n"
|
|
if err := os.WriteFile(jsonlPath, []byte(combinedContent), 0644); err != nil {
|
|
t.Fatalf("write updated JSONL failed: %v", err)
|
|
}
|
|
|
|
// Use commitToExternalBeadsRepo (don't push since no real remote)
|
|
committed, err := commitToExternalBeadsRepo(ctx, externalBeadsDir, "add ext-2", false)
|
|
if err != nil {
|
|
t.Fatalf("commitToExternalBeadsRepo failed: %v", err)
|
|
}
|
|
if !committed {
|
|
t.Error("expected commit to succeed for new issue")
|
|
}
|
|
t.Log("✓ Successfully committed issue ext-2 to external beads repo")
|
|
|
|
// Test 5: Verify commit was made in external repo (not project repo)
|
|
// Check external repo has the commit
|
|
logOutput := getGitOutputInDir(t, externalDir, "log", "--oneline", "-1")
|
|
if !strings.Contains(logOutput, "add ext-2") {
|
|
t.Errorf("external repo should have commit, got: %s", logOutput)
|
|
}
|
|
t.Log("✓ Commit correctly made in external repo")
|
|
|
|
// Test 6: Verify project repo is unchanged
|
|
projectLogOutput := getGitOutputInDir(t, projectDir, "log", "--oneline", "-1")
|
|
if strings.Contains(projectLogOutput, "add ext-2") {
|
|
t.Error("project repo should not have beads commit")
|
|
}
|
|
t.Log("✓ Project repo correctly unchanged")
|
|
|
|
// Test 7: Verify JSONL content is correct
|
|
content, err := os.ReadFile(jsonlPath)
|
|
if err != nil {
|
|
t.Fatalf("failed to read JSONL: %v", err)
|
|
}
|
|
contentStr := string(content)
|
|
if !strings.Contains(contentStr, "ext-1") || !strings.Contains(contentStr, "ext-2") {
|
|
t.Errorf("JSONL should contain both issues, got: %s", contentStr)
|
|
}
|
|
t.Log("✓ JSONL contains both issues")
|
|
|
|
t.Log("✓ External BEADS_DIR E2E test completed")
|
|
}
|
|
|
|
// TestExternalBeadsDirDetection tests various edge cases for external beads dir detection.
|
|
func TestExternalBeadsDirDetection(t *testing.T) {
|
|
ctx := context.Background()
|
|
|
|
// Store original working directory
|
|
originalWd, err := os.Getwd()
|
|
if err != nil {
|
|
t.Fatalf("failed to get original working directory: %v", err)
|
|
}
|
|
|
|
t.Run("same repo returns false", func(t *testing.T) {
|
|
// Setup a single repo
|
|
repoDir := t.TempDir()
|
|
if err := setupGitRepoInDir(t, repoDir); err != nil {
|
|
t.Fatalf("setup failed: %v", err)
|
|
}
|
|
|
|
beadsDir := filepath.Join(repoDir, ".beads")
|
|
if err := os.MkdirAll(beadsDir, 0755); err != nil {
|
|
t.Fatalf("mkdir failed: %v", err)
|
|
}
|
|
|
|
// Change to repo dir
|
|
if err := os.Chdir(repoDir); err != nil {
|
|
t.Fatalf("chdir failed: %v", err)
|
|
}
|
|
defer func() { _ = os.Chdir(originalWd) }()
|
|
git.ResetCaches()
|
|
|
|
// Same repo should return false
|
|
if isExternalBeadsDir(ctx, beadsDir) {
|
|
t.Error("isExternalBeadsDir should return false for same repo")
|
|
}
|
|
})
|
|
|
|
t.Run("different repo returns true", func(t *testing.T) {
|
|
// Setup two separate repos
|
|
projectDir := t.TempDir()
|
|
if err := setupGitRepoInDir(t, projectDir); err != nil {
|
|
t.Fatalf("setup project failed: %v", err)
|
|
}
|
|
|
|
externalDir, err := filepath.EvalSymlinks(t.TempDir())
|
|
if err != nil {
|
|
t.Fatalf("eval symlinks failed: %v", err)
|
|
}
|
|
if err := setupGitRepoInDir(t, externalDir); err != nil {
|
|
t.Fatalf("setup external failed: %v", err)
|
|
}
|
|
|
|
beadsDir := filepath.Join(externalDir, ".beads")
|
|
if err := os.MkdirAll(beadsDir, 0755); err != nil {
|
|
t.Fatalf("mkdir failed: %v", err)
|
|
}
|
|
|
|
// Change to project dir
|
|
if err := os.Chdir(projectDir); err != nil {
|
|
t.Fatalf("chdir failed: %v", err)
|
|
}
|
|
defer func() { _ = os.Chdir(originalWd) }()
|
|
git.ResetCaches()
|
|
|
|
// Different repo should return true
|
|
if !isExternalBeadsDir(ctx, beadsDir) {
|
|
t.Error("isExternalBeadsDir should return true for different repo")
|
|
}
|
|
})
|
|
|
|
t.Run("non-git directory returns false", func(t *testing.T) {
|
|
// Setup a repo for cwd
|
|
repoDir := t.TempDir()
|
|
if err := setupGitRepoInDir(t, repoDir); err != nil {
|
|
t.Fatalf("setup failed: %v", err)
|
|
}
|
|
|
|
// Non-git beads dir
|
|
nonGitDir := t.TempDir()
|
|
beadsDir := filepath.Join(nonGitDir, ".beads")
|
|
if err := os.MkdirAll(beadsDir, 0755); err != nil {
|
|
t.Fatalf("mkdir failed: %v", err)
|
|
}
|
|
|
|
// Change to repo dir
|
|
if err := os.Chdir(repoDir); err != nil {
|
|
t.Fatalf("chdir failed: %v", err)
|
|
}
|
|
defer func() { _ = os.Chdir(originalWd) }()
|
|
git.ResetCaches()
|
|
|
|
// Non-git dir should return false (can't determine, assume local)
|
|
if isExternalBeadsDir(ctx, beadsDir) {
|
|
t.Error("isExternalBeadsDir should return false for non-git directory")
|
|
}
|
|
})
|
|
}
|
|
|
|
// TestCommitToExternalBeadsRepo tests the external repo commit function.
|
|
func TestCommitToExternalBeadsRepo(t *testing.T) {
|
|
ctx := context.Background()
|
|
|
|
t.Run("commits changes to external repo", func(t *testing.T) {
|
|
// Setup external repo
|
|
externalDir, err := filepath.EvalSymlinks(t.TempDir())
|
|
if err != nil {
|
|
t.Fatalf("eval symlinks failed: %v", err)
|
|
}
|
|
if err := setupGitRepoInDir(t, externalDir); err != nil {
|
|
t.Fatalf("setup failed: %v", err)
|
|
}
|
|
|
|
beadsDir := filepath.Join(externalDir, ".beads")
|
|
if err := os.MkdirAll(beadsDir, 0755); err != nil {
|
|
t.Fatalf("mkdir failed: %v", err)
|
|
}
|
|
|
|
// Write initial JSONL
|
|
jsonlPath := filepath.Join(beadsDir, "issues.jsonl")
|
|
if err := os.WriteFile(jsonlPath, []byte(`{"id":"test-1"}`+"\n"), 0644); err != nil {
|
|
t.Fatalf("write failed: %v", err)
|
|
}
|
|
|
|
// Commit
|
|
committed, err := commitToExternalBeadsRepo(ctx, beadsDir, "test commit", false)
|
|
if err != nil {
|
|
t.Fatalf("commit failed: %v", err)
|
|
}
|
|
if !committed {
|
|
t.Error("expected commit to succeed")
|
|
}
|
|
|
|
// Verify commit exists
|
|
logOutput := getGitOutputInDir(t, externalDir, "log", "--oneline", "-1")
|
|
if !strings.Contains(logOutput, "test commit") {
|
|
t.Errorf("commit not found in log: %s", logOutput)
|
|
}
|
|
})
|
|
|
|
t.Run("returns false when no changes", func(t *testing.T) {
|
|
// Setup external repo
|
|
externalDir, err := filepath.EvalSymlinks(t.TempDir())
|
|
if err != nil {
|
|
t.Fatalf("eval symlinks failed: %v", err)
|
|
}
|
|
if err := setupGitRepoInDir(t, externalDir); err != nil {
|
|
t.Fatalf("setup failed: %v", err)
|
|
}
|
|
|
|
beadsDir := filepath.Join(externalDir, ".beads")
|
|
if err := os.MkdirAll(beadsDir, 0755); err != nil {
|
|
t.Fatalf("mkdir failed: %v", err)
|
|
}
|
|
|
|
// Write and commit JSONL
|
|
jsonlPath := filepath.Join(beadsDir, "issues.jsonl")
|
|
if err := os.WriteFile(jsonlPath, []byte(`{"id":"test-1"}`+"\n"), 0644); err != nil {
|
|
t.Fatalf("write failed: %v", err)
|
|
}
|
|
runGitInDir(t, externalDir, "add", ".beads")
|
|
runGitInDir(t, externalDir, "commit", "-m", "initial")
|
|
|
|
// Try to commit again with no changes
|
|
committed, err := commitToExternalBeadsRepo(ctx, beadsDir, "no changes", false)
|
|
if err != nil {
|
|
t.Fatalf("commit failed: %v", err)
|
|
}
|
|
if committed {
|
|
t.Error("expected no commit when no changes")
|
|
}
|
|
})
|
|
}
|
|
|
|
// Helper: Setup git repo in a specific directory (doesn't change cwd)
|
|
func setupGitRepoInDir(t *testing.T, dir string) error {
|
|
t.Helper()
|
|
|
|
// Initialize git repo
|
|
if err := exec.Command("git", "-C", dir, "init", "--initial-branch=main").Run(); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Configure git
|
|
_ = exec.Command("git", "-C", dir, "config", "user.email", "test@test.com").Run()
|
|
_ = exec.Command("git", "-C", dir, "config", "user.name", "Test User").Run()
|
|
|
|
// Create initial commit
|
|
readmePath := filepath.Join(dir, "README.md")
|
|
if err := os.WriteFile(readmePath, []byte("# Test Repo\n"), 0644); err != nil {
|
|
return err
|
|
}
|
|
if err := exec.Command("git", "-C", dir, "add", ".").Run(); err != nil {
|
|
return err
|
|
}
|
|
if err := exec.Command("git", "-C", dir, "commit", "-m", "initial commit").Run(); err != nil {
|
|
return err
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// Helper: Run git command in a specific directory
|
|
func runGitInDir(t *testing.T, dir string, args ...string) {
|
|
t.Helper()
|
|
cmdArgs := append([]string{"-C", dir}, args...)
|
|
cmd := exec.Command("git", cmdArgs...)
|
|
output, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
t.Fatalf("git %v failed: %v\n%s", args, err, output)
|
|
}
|
|
}
|
|
|
|
// Helper: Get git output from a specific directory
|
|
func getGitOutputInDir(t *testing.T, dir string, args ...string) string {
|
|
t.Helper()
|
|
cmdArgs := append([]string{"-C", dir}, args...)
|
|
cmd := exec.Command("git", cmdArgs...)
|
|
output, err := cmd.Output()
|
|
if err != nil {
|
|
t.Fatalf("git %v failed: %v", args, err)
|
|
}
|
|
return string(output)
|
|
}
|