24 Commits

Author SHA1 Message Date
Peter Chanthamynavong
dbd505656d fix(daemon): add sync-branch guard to daemon code paths (#1271)
* fix(daemon): skip export when sync-branch matches current

Prevent redundant export operations by checking if the daemon's sync
branch matches the current active branch.

Previously, the daemon would attempt to perform an export even when
already on the target branch. This logic now skips the export step in
such cases to avoid unnecessary overhead and potential conflicts.
Includes a new integration test to verify the guard logic.

* fix(daemon): prevent sync on guarded branches

Add checks to verify if a branch is guarded before performing automated
sync cycles, auto-imports, or branch-specific commit and pull operations.
This prevents the daemon from modifying protected branches or running
synchronization tasks where they are restricted.

Includes comprehensive integration tests to verify the guard logic
during sync-branch operations.

* fix(daemon): warn on sync branch misconfiguration at startup

The daemon now checks for sync branch name conflicts during its startup
loop. This provides early feedback if the sync branch is configured
in a way that might conflict with existing branches or other settings.

The warnIfSyncBranchMisconfigured function performs the validation
and logs a warning to the console. Integration tests verify that
the daemon correctly identifies and reports these misconfigurations
at initialization.
2026-01-24 17:10:08 -08:00
Peter Chanthamynavong
1561374c04 feat(sync): pull-first sync with 3-way merge (#918)
* feat(sync): implement pull-first synchronization strategy

- Add --pull-first flag and logic to sync command
- Introduce 3-way merge stub for issue synchronization
- Add concurrent edit tests for the pull-first flow

Ensures local changes are reconciled with remote updates before pushing to prevent data loss.

* feat(sync): implement 3-way merge and state tracking

- Implement 3-way merge algorithm for issue synchronization
- Add base state storage to track changes between syncs
- Add comprehensive tests for merge logic and persistence

Ensures data consistency and prevents data loss during concurrent
issue updates.

* feat(sync): implement field-level conflict merging

- Implement field-level merge logic for issue conflicts
- Add unit tests for field-level merge strategies

Reduces manual intervention by automatically resolving overlapping updates at the field level.

* refactor(sync): simplify sync flow by removing ZFC checks

The previous sync implementation relied on Zero-False-Convergence (ZFC)
staleness checks which are redundant following the transition to
structural 3-way merging. This legacy logic added complexity and
maintenance overhead without providing additional safety.

This commit introduces a streamlined sync pipeline:
- Remove ZFC staleness validation from primary sync flow
- Update safety documentation to reflect current merge strategy
- Eliminate deprecated unit tests associated with ZFC logic

These changes reduce codebase complexity while maintaining data
integrity through the robust structural 3-way merge implementation.

* feat(sync): default to pull-first sync workflow

- Set pull-first as the primary synchronization workflow
- Refactor core sync logic for better maintainability
- Update concurrent edit tests to validate 3-way merge logic

Reduces merge conflicts by ensuring local state is current before pushing changes.

* refactor(sync): clean up lint issues in merge code

- Remove unused error return from MergeIssues (never returned error)
- Use _ prefix for unused _base parameter in mergeFieldLevel
- Update callers to not expect error from MergeIssues
- Keep nolint:gosec for trusted internal file path

* test(sync): add mode compatibility and upgrade safety tests

Add tests addressing Steve's PR #918 review concerns:

- TestSyncBranchModeWithPullFirst: Verifies sync-branch config
  storage and git branch creation work with pull-first
- TestExternalBeadsDirWithPullFirst: Verifies external BEADS_DIR
  detection and pullFromExternalBeadsRepo
- TestUpgradeFromOldSync: Validates upgrade safety when
  sync_base.jsonl doesn't exist (first sync after upgrade)
- TestMergeIssuesWithBaseState: Comprehensive 3-way merge cases
- TestLabelUnionMerge: Verifies labels use union (no data loss)

Key upgrade behavior validated:
- base=nil (no sync_base.jsonl) safely handles all cases
- Local-only issues kept (StrategyLocal)
- Remote-only issues kept (StrategyRemote)
- Overlapping issues merged (LWW scalars, union labels)

* fix(sync): report line numbers for malformed JSON

Problem:
- JSON decoding errors when loading sync base state lacked line numbers
- Difficult to identify location of syntax errors in large state files

Solution:
- Include line number reporting in JSON decoder errors during state loading
- Add regression tests for malformed sync base file scenarios

Impact:
- Users receive actionable feedback for corrupted state files
- Faster troubleshooting of manual configuration errors

* fix(sync): warn on large clock skew during sync

Problem:
- Unsynchronized clocks between systems could lead to silent merge errors
- No mechanism existed to alert users of significant timestamp drift

Solution:
- Implement clock skew detection during sync merge
- Log a warning when large timestamp differences are found
- Add comprehensive unit tests for skew reporting

Impact:
- Users are alerted to potential synchronization risks
- Easier debugging of time-related merge issues

* fix(sync): defer state update until remote push succeeds

Problem:
- Base state updated before confirming remote push completion
- Failed pushes resulted in inconsistent local state tracking

Solution:
- Defer base state update until after the remote push succeeds

Impact:
- Ensures local state accurately reflects remote repository status
- Prevents state desynchronization during network or push failures

* fix(sync): prevent concurrent sync operations

Problem:
- Multiple sync processes could run simultaneously
- Overlapping operations risk data corruption and race conditions

Solution:
- Implement file-based locking using gofrs/flock
- Add integration tests to verify locking behavior

Impact:
- Guarantees execution of a single sync process at a time
- Eliminates potential for data inconsistency during sync

* docs: document sync architecture and merge model

- Detail the 3-way merge model logic
- Describe the core synchronization architecture principles

* fix(lint): explicitly ignore lock.Unlock return value

errcheck linter flagged bare defer lock.Unlock() calls. Wrap in
anonymous function with explicit _ assignment to acknowledge
intentional ignore of unlock errors during cleanup.

* fix(lint): add sync_merge.go to G304 exclusions

The loadBaseState and saveBaseState functions use file paths derived
from trusted internal sources (beadsDir parameter from config). Add
to existing G304 exclusion list for safe JSONL file operations.

* feat(sync): integrate sync-branch into pull-first flow

When sync.branch is configured, doPullFirstSync now:
- Calls PullFromSyncBranch before merge
- Calls CommitToSyncBranch after export

This ensures sync-branch mode uses the correct branch for
pull/push operations.

* test(sync): add E2E tests for sync-branch and external BEADS_DIR

Adds comprehensive end-to-end tests:
- TestSyncBranchE2E: verifies pull→merge→commit flow with remote changes
- TestExternalBeadsDirE2E: verifies sync with separate beads repository
- TestExternalBeadsDirDetection: edge cases for repo detection
- TestCommitToExternalBeadsRepo: commit handling

* refactor(sync): remove unused rollbackJSONLFromGit

Function was defined but never called. Pull-first flow saves base
state after successful push, making this safety net unnecessary.

* test(sync): add export-only mode E2E test

Add TestExportOnlySync to cover --no-pull flag which was the only
untested sync mode. This completes full mode coverage:

- Normal (pull-first): sync_test.go, sync_merge_test.go
- Sync-branch: sync_modes_test.go:TestSyncBranchE2E (PR#918)
- External BEADS_DIR: sync_external_test.go (PR#918)
- From-main: sync_branch_priority_test.go
- Local-only: sync_local_only_test.go
- Export-only: sync_modes_test.go:TestExportOnlySync (this commit)

Refs: #911

* docs(sync): add sync modes reference section

Document all 6 sync modes with triggers, flows, and use cases.
Include mode selection decision tree and test coverage matrix.

Co-authored-by: Claude <noreply@anthropic.com>

* test(sync): upgrade sync-branch E2E tests to bare repo

- Replace mocked repository with real bare repo setup
- Implement multi-machine simulation in sync tests
- Refactor test logic to handle distributed states

Coverage: sync-branch end-to-end scenarios

* test(sync): add daemon sync-branch E2E tests

- Implement E2E tests for daemon sync-branch flow
- Add test cases for force-overwrite scenarios

Coverage: daemon sync-branch workflow in cmd/bd

* docs(sync): document sync-branch paths and E2E architecture

- Describe sync-branch CLI and Daemon execution flow
- Document the end-to-end test architecture

* build(nix): update vendorHash for gofrs/flock dependency

New dependency added for file-based sync locking changes the
Go module checksum.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-07 21:27:20 -08:00
beads/crew/dave
b362b36824 feat: add session_id field to issue close/update mutations (bd-tksk)
Adds closed_by_session tracking for entity CV building per Gas Town
decision 009-session-events-architecture.md.

Changes:
- Add ClosedBySession field to Issue struct
- Add closed_by_session column to issues table (migration 034)
- Add --session flag to bd close command
- Support CLAUDE_SESSION_ID env var as fallback
- Add --session flag to bd update for status=closed
- Display closed_by_session in bd show output
- Update Storage interface to include session parameter in CloseIssue

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Executed-By: beads/crew/dave
Rig: beads
Role: crew
2025-12-31 13:14:15 -08:00
Steve Yegge
1611f16751 refactor: remove unused bd pin/unpin/hook commands (bd-x0zl)
Analysis found these commands are dead code:
- gt never calls `bd pin` - uses `bd update --status=pinned` instead
- Beads.Pin() wrapper exists but is never called
- bd hook functionality duplicated by gt mol status
- Code comment says "pinned field is cosmetic for bd hook visibility"

Removed:
- cmd/bd/pin.go
- cmd/bd/unpin.go
- cmd/bd/hook.go

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 16:02:15 -08:00
Steve Yegge
c8b912cbe6 bd sync: 2025-12-27 15:56:42 2025-12-27 15:56:42 -08:00
Jordan Hubbard
b089aaa0d6 tests: add chaos doctor repair coverage and stabilize git init
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2025-12-25 21:50:13 -04:00
Steve Yegge
e67f27c092 bd sync: 2025-12-23 23:38:57 2025-12-24 00:06:41 -08:00
Steve Yegge
2de1695615 bd sync: 2025-12-23 22:33:32 2025-12-23 22:33:33 -08:00
Steve Yegge
cf9e5a597b bd sync: 2025-12-23 20:50:50 2025-12-23 20:50:50 -08:00
Steve Yegge
9c8761abc9 bd sync: 2025-12-23 20:45:19 2025-12-23 20:45:19 -08:00
Steve Yegge
05f37d2408 Introduce slog for structured daemon logging (bd-u2sc.4)
- Replace custom daemonLogger with slog-based structured logging
- Add --log-level flag (debug, info, warn, error) with default "info"
- Add --log-json flag for JSON-formatted log output
- Update log calls to use appropriate levels (Error, Warn, Info, Debug)
- Add test helper functions (newTestLogger, newTestLoggerWithWriter)

Log output examples:
  Text: time=2025-12-23T12:30:00Z level=INFO msg="daemon started" socket=/tmp/bd.sock
  JSON: {"time":"...","level":"INFO","msg":"daemon started","socket":"/tmp/bd.sock"}

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 13:46:26 -08:00
Charles P. Cross
ee016bbb25 fix(daemon): handle diverged sync branch with fetch-rebase-retry on push (#697)
* fix(daemon): handle diverged sync branch with fetch-rebase-retry on push

When pushing to the sync branch, if the remote has newer commits that
the local worktree doesn't have, the push would fail with "fetch first"
error and the daemon would log the failure without recovery.

Bug scenario:
1. Clone A pushes commit X to sync branch
2. Clone B has local commit Y (not based on X)
3. Clone B's push fails with "fetch first" error
4. Without this fix: daemon logs failure and stops
5. With this fix: daemon fetches, rebases Y on X, and retries push

This fix adds fetch-rebase-retry logic to gitPushFromWorktree():
1. Detect push rejection due to remote having newer commits
2. Fetch the latest remote sync branch
3. Rebase local commits on top of remote
4. Retry the push

If rebase fails (e.g., due to conflicts), the rebase is aborted and
an error is returned with helpful context.

This allows multiple clones to push to the same sync branch without
manual intervention, as long as the changes don't conflict.

Adds integration test TestGitPushFromWorktree_FetchRebaseRetry that
verifies the fetch-rebase-retry behavior with diverged branches.

* fix: resolve all golangci-lint errors (cherry-pick from fix/linting-errors)

Cherry-picked linting fixes to ensure CI passes.

---------

Co-authored-by: Charles P. Cross <cpdata@users.noreply.github.com>
2025-12-22 14:17:26 -08:00
Charles P. Cross
056b989e46 Update pre-commit hook regression test for proper isolation
The TestSyncBranchCommitAndPush_WithPreCommitHook test needed fixes
to run correctly in isolation:

1. Set global dbPath variable so findJSONLPath() can locate the JSONL
   file during sync operations. Without this, the test failed with
   "JSONL path not found".

2. Add dummy git remote so hasGitRemote() returns true. The
   syncBranchCommitAndPush function skips sync branch operations
   when no remote is configured (local-only mode support).

3. Relax commit count assertion to check for "multiple commits"
   rather than exact count of 4, since sync branch initialization
   may add an extra commit depending on timing.

These changes ensure the regression test properly validates that
--no-verify bypasses pre-commit hooks in worktree commits.

Test verified:
- FAILS without --no-verify fix (confirms bug detection)
- PASSES with --no-verify fix (confirms fix works)
2025-12-11 07:11:42 -05:00
Charles P. Cross
c9dc0276aa Fix daemon sync branch commit failing with pre-commit hooks
When pre-commit hooks are installed (via "bd hooks install"), daemon auto-sync
to sync branches fails with "git commit failed in worktree: exit status 1".

Root cause:
- gitCommitInWorktree() was missing --no-verify flag
- Pre-commit hook runs "bd sync --flush-only" which fails in worktree context
- Worktree has .beads directory, triggering hook execution
- Hook fails because bd cannot find proper database in worktree path

The fix adds --no-verify to git commit in gitCommitInWorktree(), matching
the existing implementation in internal/syncbranch/worktree.go line 684.

This is correct because:
- Worktree commits are internal to bd sync operations
- Running pre-commit hooks in worktree context is semantically wrong
- The library function already skips hooks for this reason

Includes regression test that:
- Creates a repo with sync branch configured
- Installs a failing pre-commit hook (simulating bd hook behavior)
- Verifies commits succeed because --no-verify bypasses the hook
- Tests multiple consecutive commits to ensure consistent behavior

Tested manually by:
1. Creating issue with "bd create" (triggers mutation event)
2. Verifying daemon logs show successful commit to sync branch
3. Confirming push to remote sync branch completes
2025-12-09 22:17:23 -05:00
Abhinav Gupta
ac8ef9b9e3 test: replace manual os.Chdir with t.Chdir in tests (#457)
Replaces manual working directory save/restore patterns
with Go's built-in `t.Chdir()` helper across 23 test files.

The manual pattern involved calling `os.Getwd()` to save
the original directory, using `defer os.Chdir(origWd)` for
restoration, and manually handling errors during directory
changes. This boilerplate has been replaced with single
`t.Chdir(path)` calls that handle cleanup automatically.

The `t.Chdir()` method automatically restores the working
directory when the test completes, eliminating the need for
manual defer statements and error handling.

Total:
~75 instances replaced (assuming Claude's math is right)

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-04 11:21:43 -08:00
Steve Yegge
0dc8452c56 feat: Respect BEADS_SYNC_BRANCH environment variable
Fixes daemon and bd sync to honor BEADS_SYNC_BRANCH environment variable
as documented in PROTECTED_BRANCHES.md for CI/CD temporary overrides.

Changes:
- Updated internal/syncbranch.Get() to prioritize env var over DB config
- Both daemon sync and bd sync CLI now use syncbranch.Get()
- Added comprehensive tests for env var override behavior
- Validates branch names using git-style rules

This enables CI/CD workflows to override sync branch per-job without
mutating database config.

Based on PR #364 by Charles P. Cross <cpdata@users.noreply.github.com>
Co-authored-by: Charles P. Cross <cpdata@users.noreply.github.com>
2025-11-22 18:17:19 -08:00
Steve Yegge
57253f93a3 Context propagation with graceful cancellation (bd-rtp, bd-yb8, bd-2o2)
Complete implementation of signal-aware context propagation for graceful
cancellation across all commands and storage operations.

Key changes:

1. Signal-aware contexts (bd-rtp):
   - Added rootCtx/rootCancel in main.go using signal.NotifyContext()
   - Set up in PersistentPreRun, cancelled in PersistentPostRun
   - Daemon uses same pattern in runDaemonLoop()
   - Handles SIGINT/SIGTERM for graceful shutdown

2. Context propagation (bd-yb8):
   - All commands now use rootCtx instead of context.Background()
   - sqlite.New() receives context for cancellable operations
   - Database operations respect context cancellation
   - Storage layer propagates context through all queries

3. Cancellation tests (bd-2o2):
   - Added import_cancellation_test.go with comprehensive tests
   - Added export cancellation test in export_test.go
   - Tests verify database integrity after cancellation
   - All cancellation tests passing

Fixes applied during review:
   - Fixed rootCtx lifecycle (removed premature defer from PersistentPreRun)
   - Fixed test context contamination (reset rootCtx in test cleanup)
   - Fixed export tests missing context setup

Impact:
   - Pressing Ctrl+C during import/export now cancels gracefully
   - No database corruption or hanging transactions
   - Clean shutdown of all operations

Tested:
   - go build ./cmd/bd ✓
   - go test ./cmd/bd -run TestImportCancellation ✓
   - go test ./cmd/bd -run TestExportCommand ✓
   - Manual Ctrl+C testing verified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 21:57:23 -05:00
Steve Yegge
0f4b03e262 Optimize test suite: split integration tests, add -short support
- Split slow importer integration tests into separate file
- Add t.Short() guards to 10 slow daemon tests
- Document test organization in TEST_OPTIMIZATION.md
- Fast tests now run in ~50s vs 3+ minutes
- Use 'go test -short ./...' for fast feedback

Amp-Thread-ID: https://ampcode.com/threads/T-29ae21ac-749d-43d7-bf0c-2c5f7a06ae76
Co-authored-by: Amp <amp@ampcode.com>
2025-11-05 20:39:47 -08:00
Steve Yegge
94e71642f9 fix: increase Windows filesystem delays in flaky tests
- TestDaemonAutoImportAfterGitPull: 200ms -> 500ms for filesystem timestamp precision
- TestSyncBranchPull_Success: 100ms -> 300ms for file I/O settling
- Also increase Linux delays slightly for more reliability
2025-11-04 10:20:12 -08:00
Steve Yegge
859fbd4976 fix: resolve CI test failures on Linux and Windows
- Fix TestMigrateCommand: Use correct sqlite3 driver name and file URI format
- Fix TestDaemonAutoImportAfterGitPull: Add Windows-specific delay for filesystem timestamp precision
- Fix TestSyncBranchPull_Success: Add Windows-specific delay for file I/O settling

Amp-Thread-ID: https://ampcode.com/threads/T-5abd1f2f-9607-4111-af9d-19df64362ac7
Co-authored-by: Amp <amp@ampcode.com>
2025-11-04 10:02:08 -08:00
Steve Yegge
68876dd98f fix: CI failures - performance thresholds, test eligibility, Nix hash, lint errors
- TestSyncBranchPerformance: Increase Windows threshold to 500ms (was 150ms)
  Windows git operations are ~3x slower than Unix
- TestCompactTier1: Fix eligibility by using 7-day minimum and 8-day closure
  Changed compact_tier1_days from 0 to 7 to properly test eligibility checks
- Nix flake: Update vendorHash for current go.mod dependencies
  sha256-cS2saiyKMgw4cXSc2INBHNJfJz5300ybI6Vxda1vLGk=
- Lint fixes:
  - Remove unused 'quiet' parameter from createConfigYaml
  - Change template file permissions from 0644 to 0600 (gosec G306)
  - Add nosec comment for sanitized file path (gosec G304)
2025-11-03 22:01:37 -08:00
Steve Yegge
86c645603e Fix all gosec, misspell, and unparam linter errors
- Add #nosec directives with explanations for all gosec warnings in worktree operations
- Tighten directory permissions from 0755 to 0750 for better security
- Fix misspellings: archaeological -> archeological, cancelled -> canceled
- Remove unused jsonlPath parameter from syncBranchCommitAndPush
- Change branchExists to return bool instead of (bool, error) - error was never used

All changes maintain backward compatibility and improve code quality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 20:06:05 -08:00
Steve Yegge
c5b2fbbc9d Add comprehensive sync-branch test coverage (bd-0e74)
Added 4 new integration tests to ensure sync-branch workflow is robust:

1. TestSyncBranchConfigChange - Validates changing sync.branch config
   after worktrees exist. Tests smooth transition between branches.

2. TestSyncBranchMultipleConcurrentClones - Tests 3-way clone sync
   workflow. Simulates real multi-agent collaboration scenario.

3. TestSyncBranchPerformance - Validates commit overhead < 150ms.
   Current performance: avg 77ms (well within target).

4. TestSyncBranchNetworkFailure - Tests graceful handling of network
   errors during push. Ensures local commits succeed even when remote
   is unreachable.

All tests pass:
- cmd/bd (sync branch): PASS (5.361s)
- internal/git: PASS (1.071s)
- internal/syncbranch: PASS (0.312s)

Test coverage now includes: config change handling, multiple concurrent
clones, network failure recovery, performance validation, fresh setup,
issue operations, error handling, sparse checkout.

Closes bd-0e74

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 18:30:38 -08:00
Steve Yegge
712fb772fe Complete sync branch daemon tests (bd-7bd2, bd-502e)
- Add daemon_sync_branch.go with syncBranchCommitAndPush/Pull functions
- Add daemon_sync_branch_test.go with 7 comprehensive tests
- All tests passing: NotConfigured, Success, NoChanges, WorktreeHealthCheck, Pull, EndToEnd
- Key fix: initMainBranch must run BEFORE creating issues/JSONL
- Close bd-7bd2 and bd-502e

Amp-Thread-ID: https://ampcode.com/threads/T-e3d7ba22-99d1-4210-a6db-1dcc3bdd622b
Co-authored-by: Amp <amp@ampcode.com>
2025-11-02 16:40:21 -08:00