beads/BD-3S8-CHANGES.md

# BD-3S8: Multi-Clone Sync Fix

## Problem

When multiple clones of a repository both commit to the `beads-sync` branch and one tries to pull, git's merge would fail due to diverged histories. This made multi-clone workflows unreliable.

## Solution

Replace git's commit-level merge with a content-based merge that handles divergence gracefully:

1. **Fetch** (not pull) from remote
2. **Detect divergence** using `git rev-list --left-right --count`
3. **Extract JSONL** from merge base, local HEAD, and remote
4. **Merge content** using bd's 3-way merge algorithm
5. **Reset to remote's history** (adopt their commit graph)
6. **Commit merged content** on top

This ensures sync never fails due to git merge conflicts - we handle merging at the JSONL content level where we have semantic understanding of the data.

## Changes

### `internal/syncbranch/worktree.go`

**New functions:**
- `getDivergence()` - Detects how many commits local/remote are ahead/behind
- `performContentMerge()` - Extracts and merges JSONL content from base/local/remote
- `performDeletionsMerge()` - Merges deletions.jsonl by union (keeps all deletions)
- `extractJSONLFromCommit()` - Extracts file content from a specific git commit
- `copyJSONLToMainRepo()` - Refactored helper for copying JSONL files
- `preemptiveFetchAndFastForward()` - Reduces divergence by fetching before commit

**Modified functions:**
- `PullFromSyncBranch()` - Now handles three cases:
  - Already up-to-date: Remote has nothing new
  - Fast-forward: Simple `--ff-only` merge
  - **Diverged**: Content-based merge (the fix)
- `CommitToSyncBranch()` - Now fetches and fast-forwards before committing

**Enhanced structs:**
- `PullResult` - Added `Merged` and `FastForwarded` fields

### `cmd/bd/sync.go`

- Updated output messages to show merge type (fast-forward vs merged divergent histories)

### `internal/syncbranch/worktree_divergence_test.go` (new file)

Test coverage for:
- `getDivergence()` - 4 scenarios
- `extractJSONLFromCommit()` - 3 scenarios
- `performContentMerge()` - 2 scenarios
- `performDeletionsMerge()` - 2 scenarios

## How It Works

```
Clone A commits and pushes: origin/beads-sync = A -- B -- C
Clone B commits locally:     local beads-sync = A -- B -- D

When Clone B syncs:
1. Fetch: gets C from origin
2. Detect divergence: local ahead 1, remote ahead 1
3. Find merge base: B
4. Extract: base=B's JSONL, local=D's JSONL, remote=C's JSONL
5. Content merge: merge JSONL using 3-way algorithm
6. Reset to origin: beads-sync = A -- B -- C
7. Commit merged: beads-sync = A -- B -- C -- M (merged content)
8. Push: no conflict, linear history
```

## Merge Rules

The 3-way merge uses these rules (from `internal/merge/merge.go`):

- **New issues**: Added from both sides
- **Deleted issues**: Deletion wins over modification
- **Modified issues**: Field-level merge
  - `status`: "closed" always wins over "open"
  - `updated_at`: Takes the max (latest)
  - `closed_at`: Only set if status is "closed"
  - `dependencies`: Union of both sides
  - Other fields: Standard 3-way merge

## Edge Cases Handled

1. **Remote branch doesn't exist** - Nothing to pull, return early
2. **No common ancestor** - Use empty base for merge
3. **File doesn't exist in commit** - Use empty content
4. **Deletions.jsonl missing** - Non-fatal, skip deletion merge
5. **True conflicts** - Currently fails with error (manual resolution required)

## Future Improvements

### 1. Auto-Resolve All Conflicts (No Manual Resolution Required)

Currently, true conflicts (both sides changed same field to different values) fail the sync. This should be changed to auto-resolve deterministically:

| Field | Auto-Resolution Strategy |
|-------|-------------------------|
| `updated_at` | Already handled - takes max (latest) |
| `closed_at` | Already handled - takes max (latest) |
| `status` | Already handled - "closed" wins |
| `Priority` | Take higher priority (lower number = more urgent) |
| `IssueType` | Take left (local wins) |
| `Notes` | **Concatenate both** with separator (preserves all contributions) |
| `Title` | Take from side with latest `updated_at` on the issue |
| `Description` | Take from side with latest `updated_at` on the issue |

With this strategy, **no conflicts ever require manual resolution** - there's always a deterministic auto-resolution. The merge driver becomes fully automatic.

### 2. Auto-Push After Merge (Default Behavior)

Users shouldn't need to review merge diffs on beads metadata. The goal is "one command that just works":

```
bd sync  # Should handle everything, including push
```

**Proposed behavior:**
- After successful content merge, auto-push by default
- Only hold off on push when unsafe conditions detected

**Safety checks before auto-push:**
1. No conflict markers in JSONL (shouldn't happen with full auto-resolve)
2. Issue count sanity check - didn't drop to zero unexpectedly
3. Reasonable deletion threshold - didn't delete > N% of issues in one sync

**The deletions manifest problem:**
- In multi-clone environments, deletions from one clone propagate to others
- This is correct behavior, but can feel like "corruption" when unexpected
- Swarms legitimately close/delete all issues sometimes
- Hard to distinguish "swarm finished all work" from "corruption"

**Proposed safeguards:**
- Track whether issues were *closed* (status change) vs *deleted* (removed from JSONL)
- Closing all issues = legitimate (swarm finished)
- Deleting all issues when there were many = suspicious, pause for confirmation
- Config option: `sync.auto_push` (default: true, can set to false for paranoid mode)

**Integration with bd doctor:**
- `bd doctor --fix` should also run this recovery logic
- But `bd doctor` is for daily/upgrade maintenance, not inner loop
- `bd sync` must handle divergence recovery itself

**The "one nuclear fix" philosophy:**
- `bd sync` should just work 99.9% of the time
- Auto-resolve all conflicts
- Auto-push when safe
- Only fail/pause when genuinely dangerous (mass deletion detected)

### 3. V1 Implementation Plan

Keep it simple for the first iteration:

**Auto-push behavior:**
1. After successful content merge, auto-push by default
2. One safety check: if issue count dropped by >50% AND there were >5 issues before, log a warning but still push
3. Config option `sync.require_confirmation_on_mass_delete` (default: false) for paranoid users who want to be prompted

**Rationale:**
- Logging gives forensics if something goes wrong
- Doesn't block the happy path (99.9% of syncs)
- Users who've been burned can enable confirmation mode
- We can tighten safeguards later based on real-world feedback

**What "mass deletion" means:**
- Issues that **vanished** from `issues.jsonl` (not just closed)
- `status=closed` is fine - swarm finished legitimately
- Issues disappearing entirely is suspicious

**Future safeguards (not v1):**
- Tombstone TTL: Ignore deletions older than N days
- Deletion rate limit: Pause if deletions.jsonl suddenly has 100+ new entries
- Protected issues: Certain issues can't be deleted via sync

---

## Summary of Work Items

1. **Already implemented (this PR):**
   - Content-based merge for diverged histories
   - Pre-emptive fetch before commit
   - Deletions.jsonl merge
   - Fast-forward detection

2. **Still to implement:**
   - Auto-resolve all field conflicts (no manual resolution)
   - Auto-push after merge with safety check
   - Mass deletion warning/logging
   - Config option for confirmation mode