The `bd update --type` command was rejecting custom types (like role, agent,
rig) even when configured via types.custom. The issue was that type validation
used IsValid() which only checks the 5 core types, ignoring custom types.
Changes:
- CLI (update.go): Use IsValidWithCustom() with types from config
- Storage (validators.go): Add validateIssueTypeWithCustom() function
- Storage (queries.go, transaction.go): Fetch and pass custom types
The error message now dynamically shows all valid types including custom ones.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The original PR added retry logic on top of BEGIN IMMEDIATE, but this caused
multi-minute hangs because:
1. Connection has busy_timeout=30s set via pragma
2. Each BEGIN IMMEDIATE waits up to 30s before returning SQLITE_BUSY
3. With 5 retries, worst case was 5 × 30s = 150+ seconds
The fix removes the retry loop since SQLite's busy_timeout already handles
retries internally. BEGIN IMMEDIATE still acquires the write lock early,
preventing deadlocks - we just let busy_timeout handle contention.
Root cause analysis in bd-9ldm.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update withTx to use BEGIN IMMEDIATE with exponential backoff retry
on SQLITE_BUSY errors. This prevents "database is locked" failures
during concurrent operations (daemon + CLI, multi-agent workflows).
Changes:
- withTx now uses beginImmediateWithRetry (same pattern as RunInTransaction)
- Add dbExecutor interface for helper functions that work with both
*sql.Tx and *sql.Conn
- Update all withTx callers to use *sql.Conn
- Refactor DeleteIssue to use withTx (fixes the specific error in auto-import)
- Update markIssuesDirtyTx to accept dbExecutor interface
Affected paths:
- MarkIssuesDirty, ClearDirtyIssuesByID (dirty.go)
- AddDependency, RemoveDependency (dependencies.go)
- executeLabelOperation (labels.go)
- AddComment (events.go)
- ApplyCompaction (compact.go)
- DeleteIssue (queries.go)
Note: Some direct BeginTx calls in queries.go (CloseIssue, UpdateIssue,
ReopenIssue, DeleteIssues) still use the old pattern and could be
refactored in a follow-up.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds Dolt auto-commit functionality for write commands and sets explicit commit authors.
Includes fix for race condition in commandDidWrite (converted to atomic.Bool).
Original PR: #1267 by @coffeegoddd
Co-authored-by: Dustin Brown <dustin@dolthub.com>
Epics now require a "Working Model" section in their description,
in addition to "Success Criteria". This provides clear guidance on
HOW the epic will be executed:
- Owner role: Coordinator vs Implementer
- Delegation target: Polecats, crew, external
- Review process: Approval gates
Closes gt-0lp
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds operational improvements to the Dolt storage backend for increased
reliability in production environments with concurrent clients:
1. Lock retry with exponential backoff:
- Add LockRetries and LockRetryDelay config options
- Automatic retry on lock contention (default: 30 retries, ~6s window)
- Exponential backoff starting at 100ms
- Handles transient format version errors during manifest updates
2. Stale lock file cleanup:
- Detect and clean orphaned .dolt/noms/LOCK files on startup
- Prevents "database is read only" errors after crashes
- Only removes empty locks older than 5 seconds
3. Transient error detection:
- isTransientDoltError() detects retryable conditions
- isLockError() identifies lock contention scenarios
- cleanupStaleDoltLock() safely removes orphaned locks
These improvements address common issues in multi-process environments
where the Dolt embedded driver creates exclusive locks that persist
after unexpected termination.
Co-authored-by: upstream_syncer <matthew.baker@pihealth.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
When creating issues with explicit IDs via PrefixOverride (for cross-rig
creation), the storage layer was still validating against the local
config prefix, causing errors like:
issue ID 'da-DataArchive-witness' does not match configured prefix 'hq'
The queries.go layer already handled PrefixOverride correctly, but
transaction.go did not. This fix adds the same PrefixOverride handling
to transaction.go:
1. Check if PrefixOverride is set
2. If set, use it as the prefix and skip validation (caller knows best)
3. Otherwise, fall back to existing IDPrefix/configPrefix logic
This allows gt doctor --fix to create agent beads for non-local rigs.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
fix(daemon): prevent zombie state after database file replacement
Adds checkFreshness() to health check paths (GetMetadata, GetConfig, GetAllConfig) and refactors reconnect() to validate new connection before closing old.
PR-URL: https://github.com/steveyegge/beads/pull/1213
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(autoimport): auto-correct deleted status to tombstone for JSONL compatibility (GH#1223)
This fix addresses the 'Stuck in sync diversion loop' issue where v0.48.0
encountered validation errors during JSONL import. The issue occurs when
JSONL files from older versions have issues with status='deleted' but the
current code expects status='tombstone' for deleted issues.
Changes:
- Add migration logic in parseJSONL to auto-correct 'deleted' status to 'tombstone'
- Ensure tombstones always have deleted_at timestamp set
- Add debug logging for both migration operations
- Prevents users from being stuck in sync divergence when upgrading
Fixes GH#1223: Stuck in sync diversion loop
* fix(autoimport): comprehensively fix corrupted deleted_at on non-tombstone issues (GH#1223)
The initial fix for GH#1223 only caught issues with status='deleted', but the real
data in the wild had issues with status='closed' (or other statuses) but also
had deleted_at set, which violates the validation rule.
Changes:
- Add broader migration logic: any non-tombstone issue with deleted_at should become tombstone
- Apply fix in all three JSONL parsing locations:
- internal/autoimport/autoimport.go (parseJSONL for auto-import)
- cmd/bd/import.go (import command)
- cmd/bd/daemon_sync.go (daemon sync helper)
- Add comprehensive test case for corrupted closed issues with deleted_at
- Fixes the 'non-tombstone issues cannot have deleted_at timestamp' validation error
during fresh bd init or import
Fixes GH#1223: Stuck in sync diversion loop
* Add merge driver comment to .gitattributes
* fix: properly clean up .gitattributes during bd admin reset
Fixes GH#1223 - Stuck in sync diversion loop
The removeGitattributesEntry() function was not properly cleaning up
beads-related entries from .gitattributes. It only removed lines
containing "merge=beads" but left behind:
- The comment line "# Use bd merge for beads JSONL files"
- Empty lines following removed entries
This caused .gitattributes to remain in a modified state after
bd admin reset --force, triggering sync divergence warning loop.
The fix now:
- Skips lines containing "merge=beads" (existing behavior)
- Skips beads-related comment lines
- Skips empty lines that follow removed beads entries
- Properly cleans up file so it's either empty (and gets deleted)
or contains only non-beads content
---------
Co-authored-by: Amp <amp@example.com>
The Dolt storage was scanning created_at and updated_at directly into
time.Time fields, but SQLite stores these as TEXT strings. The Go SQLite
driver cannot automatically convert TEXT to time.Time.
Added parseTimeString() helper and fixed all scan functions:
- issues.go: scanIssue()
- dependencies.go: scanIssueRow()
- history.go: GetIssueHistory(), GetIssueAsOf()
- transaction.go: scanIssueTx()
Fixes bd-4dqmy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move LocalProvider from cmd/bd/doctor/git.go to internal/storage/local_provider.go
where it belongs alongside StorageProvider. Both implement IssueProvider for
orphan detection - LocalProvider for direct SQLite access (--db flag),
StorageProvider for the global Storage interface.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(orphans): honor --db flag for cross-repo orphan detection
Problem:
- `bd orphans --db /path` ignored the --db flag entirely
- FindOrphanedIssues() hardcoded local .beads/ directory
Solution:
- Introduce IssueProvider interface for abstract issue lookup
- Add StorageProvider adapter wrapping Storage instances
- Update FindOrphanedIssues to accept provider instead of path
- Wire orphans command to create provider from --db flag
Closes: steveyegge/beads#1196
* test(orphans): add cross-repo and provider tests for --db flag fix
- Add TestFindOrphanedIssues_WithMockProvider (table-driven, UT-01 through UT-09)
- Add TestFindOrphanedIssues_CrossRepo (validates --db flag honored)
- Add TestFindOrphanedIssues_LocalProvider (backward compat RT-01)
- Add TestFindOrphanedIssues_ProviderError (error handling UT-07)
- Add TestFindOrphanedIssues_IntegrationCrossRepo (IT-02 full)
- Add TestLocalProvider_* unit tests
Coverage for IssueProvider interface and cross-repo orphan detection.
* docs: add bd orphans command to CLI reference
Document the orphan detection command including the cross-repo
workflow enabled by the --db flag fix in this PR.
Implements configurable per-field merge strategies (hq-ew1mbr.11):
- Add FieldStrategy type with strategies: newest, max, union, manual
- Add conflict.fields config section for per-field overrides
- compaction_level defaults to "max" (highest value wins)
- estimated_minutes defaults to "manual" (flags for user resolution)
- labels defaults to "union" (set merge)
Manual conflicts are displayed during sync with resolution options:
bd sync --ours / --theirs, or bd resolve <id> <field> <value>
Config example:
conflict:
strategy: newest
fields:
compaction_level: max
estimated_minutes: manual
labels: union
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(ready): exclude molecule steps from bd ready by default (GH#1239)
Add ID prefix constants (IDPrefixMol, IDPrefixWisp) to types.go as single
source of truth. Update pour.go and wisp.go to use these constants.
GetReadyWork now excludes issues with -mol- in their ID when no explicit
type filter is specified. Users can still see mol steps with --type=task.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ready): config-driven ID pattern exclusion (GH#1239)
Add ready.exclude_id_patterns config for excluding IDs from bd ready.
Default patterns: -mol-, -wisp- (molecule steps and wisps).
Changes:
- Add IncludeMolSteps to WorkFilter for internal callers
- Update findGateReadyMolecules and getMoleculeCurrentStep to use it
- Make exclusion patterns config-driven via ready.exclude_id_patterns
- Remove hardcoded MolStepIDPattern() in favor of config
- Add test for custom patterns (e.g., gastown's -role-)
Usage: bd config set ready.exclude_id_patterns "-mol-,-wisp-,-role-"
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: remove -role- example from ready.go comments
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: remove GH#1239 references from code comments
Issue references belong in commit messages, not code.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add integration tests for the Gas Town hub+spokes topology where:
- Hub (mayor rig) runs dolt sql-server with remotesapi
- Spokes (crew clones) configure hub as their only peer
- Data flows through hub in star topology
Tests included:
- TestHubSpokes_MultiCloneSync: Basic two-spoke convergence
- TestHubSpokes_WorkDispatch: Hub dispatches work, spokes complete it
- TestHubSpokes_ThreeSpokesConverge: Three-spoke convergence test
Note: These tests follow the same pattern as peer_sync_integration_test.go
and document current behavior including the no common ancestor limitation
that affects cross-database sync.
Closes bd-phwci
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes stack overflow and database initialization failures when running bd on WSL2 with Docker Desktop bind mounts.
## Problem
The bd CLI crashed with stack overflow when running on WSL2 with repositories on Docker Desktop bind mounts (/mnt/wsl/docker-desktop-bind-mounts/...). SQLite WAL mode fails with 'locking protocol' error on these network filesystems.
## Solution
- Expand WAL mode detection to identify Docker bind mounts at /mnt/wsl/* (in addition to Windows paths at /mnt/[a-zA-Z]/)
- Fall back to DELETE journal mode on these problematic paths
- Add comprehensive unit tests for path detection
Fixes GH #1224, relates to GH #920
Co-authored-by: maphew <matt.wilkie@gmail.com>
Remove Gas Town-specific type constants (TypeMolecule, TypeGate, TypeConvoy,
TypeMergeRequest, TypeSlot, TypeAgent, TypeRole, TypeRig, TypeEvent, TypeMessage)
from internal/types/types.go.
Beads now only has core work types built-in:
- bug, feature, task, epic, chore
All Gas Town types are now purely custom types with no special handling in beads.
Use string literals like "gate" or "molecule" when needed, and configure
types.custom in config.yaml for validation.
Changes:
- Remove Gas Town type constants from types.go
- Remove mr/mol aliases from Normalize()
- Update bd types command to only show core types
- Replace all constant usages with string literals throughout codebase
- Update tests to use string literals
This decouples beads from Gas Town, making it a generic issue tracker.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Improve the federation status command to show more comprehensive
information similar to `git remote -v` with health info:
- Show peer URLs alongside peer names
- Display pending local changes count (uncommitted)
- Test connectivity to each peer (via fetch)
- Track and display last sync time in metadata table
- Show reachability status with error messages on failure
Changes:
- cmd/bd/federation.go: Enhanced status output with URLs, connectivity
checks, pending changes, and last sync time
- internal/storage/dolt/federation.go: Added getLastSyncTime/setLastSyncTime
methods using metadata table, record sync time on successful sync
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
During bd init, auto-import fails with "invalid issue type" errors even
when types.custom is defined in config.yaml. This happens because custom
types are read from the database, but the database is being created
during init and doesn't have the config set yet.
Changes:
- Add GetCustomTypesFromYAML() to internal/config/config.go to read
types.custom from config.yaml via viper
- Modify GetCustomTypes() in sqlite/config.go to fallback to config.yaml
when the database doesn't have types.custom configured
- Add tests for GetCustomTypesFromYAML()
This allows fresh clones with custom types defined in config.yaml (e.g.,
Gas Town types like molecule, gate, convoy, agent, event) to successfully
auto-import their JSONL during bd init.
Fixes GH#1225
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Merge SQL user authentication with Emma federation sync implementation:
- Add federation_peers table for encrypted credential storage
- Add credentials.go with AES-256-GCM encryption, SHA-256 key derivation
- Extend FederatedStorage interface with credential methods
- Add --user, --password, --sovereignty flags to bd federation add-peer
- Integrate credentials into PushTo/PullFrom/Fetch via withPeerCredentials
- DOLT_REMOTE_USER/PASSWORD env vars protected by mutex for concurrency
Credentials automatically used when syncing with peers that have stored auth.
Continues: bd-wkumz.10, Closes: bd-4p67y
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add --federation-port and --remotesapi-port flags (default 3306/8080)
- Fix log file leak in server.go - track and close on Stop()
- Add BEADS_DOLT_PASSWORD env var for server mode authentication
- Update DSN to include password when set
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add --federation flag to bd daemon start that runs dolt sql-server
instead of the embedded driver. Enables multi-writer support and
exposes remotesapi on port 8080 for peer-to-peer push/pull.
Changes:
- Add --federation flag to daemon start command
- Create dolt server manager (internal/storage/dolt/server.go)
- Update DoltStore to support server mode via MySQL protocol
- Integrate server lifecycle into daemon (auto-start/stop)
- Add tests for server management and server mode connections
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The ncruces/go-sqlite3 driver does not always auto-convert TEXT columns
to time.Time. This caused scan errors on updated_at/created_at fields,
blocking witness startup.
Fix: Scan timestamps into sql.NullString and parse with parseTimeString()
helper that handles RFC3339Nano, RFC3339, and SQLite native formats.
Fixes: bd-4dqmy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add support for importing routes.jsonl and interactions.jsonl during
Dolt bootstrap. Previously only issues.jsonl was imported.
Changes:
- Add routes and interactions tables to Dolt schema
- Import routes before issues (no dependencies)
- Import interactions after issues (may reference issue_id)
- Reuse audit.Entry type instead of duplicating
- Add tests for multi-file bootstrap
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add versioned storage methods for incremental export support:
- GetChangesSinceExport: returns changes since a commit hash, with
NeedsFullExport flag for invalid/GC'd commits
- CommitExists: checks if a commit hash exists, supports short prefixes
Also fixes dolt_diff syntax for embedded driver (from_ref, to_ref, table).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add peer-to-peer synchronization for Dolt-backed beads databases:
- New FederatedStorage interface with PushTo, PullFrom, Fetch, ListRemotes,
RemoveRemote, and SyncStatus methods
- DoltStore implementation using DOLT_PUSH, DOLT_PULL, DOLT_FETCH
- Full bd federation command with subcommands:
- sync: bidirectional sync with conflict resolution (--strategy ours|theirs)
- status: show ahead/behind counts and conflict state
- add-peer/remove-peer/list-peers: manage federation remotes
- Comprehensive tests for all federation APIs
Closes: bd-wkumz.4
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add detection and hook support for jujutsu repositories:
- IsJujutsuRepo(): detects .jj directory
- IsColocatedJJGit(): detects colocated jj+git repos
- GetJujutsuRoot(): finds jj repo root
For colocated repos (jj git init --colocate):
- Install simplified hooks without staging (jj auto-commits working copy)
- Worktree handling preserved for git worktrees in colocated repos
For pure jj repos (no git):
- Print alias instructions since jj doesn't have native hooks yet
Closes: hq-ew1mbr.12
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add integration tests validating Dolt APIs needed for peer-to-peer
federation between Gas Towns:
- TestFederationDatabaseIsolation: verify separate towns have isolated DBs
- TestFederationVersionControlAPIs: branch, checkout, merge, commit
- TestFederationRemoteConfiguration: AddRemote accepts dolthub:// URLs
- TestFederationHistoryQueries: commit history, dolt_log, AS OF
Key finding: Production federation uses dolt sql-server remotesapi
(port 8080) for direct peer-to-peer push/pull - no DoltHub required.
Also fixes concurrent_test.go Merge() return value handling.
Refs: bd-wkumz (Federation epic), hq-ew1mbr.7 (prototype task)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(sync): use sync-branch worktree for --full --no-pull (#1173)
Bug 1: PullFromSyncBranch was copying uncommitted worktree changes to
main repo when remoteAhead==0. This corrupted the 3-way merge because
local changes appeared as remote changes. Fixed by copying only the
committed state from HEAD instead of the working directory.
Bug 2: doExportOnlySync was checking main repo for changes via
gitHasBeadsChanges, but when sync-branch is configured, changes go to
the worktree, not main. Fixed by detecting sync-branch config and using
CommitToSyncBranch which operates on the worktree.
Fixes#1173
* refactor(sync): consolidate sync-branch detection and commit/push logic
Extract repeated patterns into reusable helpers:
- SyncBranchContext struct: holds branch name and repo root
- getSyncBranchContext(): detects sync-branch config from store
- commitAndPushBeads(): handles both sync-branch and regular git workflows
This eliminates duplicated sync-branch detection code (was in 3 places)
and the duplicated commit/push conditional logic (was in 2 places).
Net reduction of ~20 lines while improving maintainability.
* fix: remove unused bool return from commitAndPushBeads
Change routing.mode default from "auto" to "" (empty/disabled).
This fixes GH#1165 where fresh `bd init --prefix X` followed by
`bd create` would unexpectedly route to ~/.beads-planning and fail
with "database not initialized: issue_prefix config is missing".
Auto-routing now requires explicit opt-in via:
- `bd init --contributor` flag, OR
- `bd config set routing.mode auto`
Includes test verifying the default and doc updates clarifying
the opt-in requirement.
Code review improvements to internal/config/sync.go:
1. Warning suppression toggle
- Add ConfigWarnings bool to enable/disable warnings
- Add ConfigWarningWriter io.Writer for testable output
2. Consolidate sync mode constants
- cmd/bd/sync_mode.go now imports from internal/config
- Single source of truth for mode values
- Uses shared IsValidSyncMode() for validation
3. Fix empty sovereignty semantics
- Empty now returns SovereigntyNone (no restriction)
- Only non-empty invalid values fall back to T1 with warning
4. Export validation helpers
- IsValidSyncMode(), IsValidConflictStrategy(), IsValidSovereignty()
- ValidSyncModes(), ValidConflictStrategies(), ValidSovereigntyTiers()
- String() methods on all typed values
5. Logger interface
- ConfigWarningWriter allows custom logging destinations
- Tests can capture warnings without os.Stderr manipulation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Commit 356ab92b deleted internal/config/sync.go while adding
cmd/bd/sync_mode.go, but config.go still references the types and
functions from sync.go, causing build failures.
These serve different purposes:
- internal/config/sync.go: viper-based (config.yaml), has warnings
- cmd/bd/sync_mode.go: storage-based (database), no warnings
Both should coexist. This restores sync.go and sync_test.go from
e82e15a8, and fixes type conversion in sync.go:805.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#1076.
When git worktrees have sparse-checkout enabled, git add operations can
fail with 'outside of sparse-checkout cone' errors. Adding the --sparse
flag allows git to stage files outside the sparse-checkout cone.
Changes:
- Add --sparse flag to git add in daemon_sync_branch.go (line 165)
- Add --sparse flag to git add in migrate_sync.go (line 343)
- Add --sparse flag to git add in sync_branch.go (line 308)
- Add --sparse flag to git add in worktree.go (line 732)
All changes include comments referencing #1076 for traceability.
The scanIssues and scanIssuesWithDependencyType helper functions were
not checking rows.Err() after iterating through query results. This
could cause errors during iteration (connection drops, context
cancellation, etc.) to be silently ignored.
Per Go database/sql best practices, rows.Err() should always be checked
after a rows.Next() loop completes to catch any errors that occurred
during iteration.
Co-authored-by: Steven Syrek <steven.syrek@deepl.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(config): remove duplicate declarations between config.go and sync.go
Commit e82e15a8 created sync.go with typed constants (SyncMode,
ConflictStrategy, Sovereignty) but didn't remove the original untyped
constants from config.go that were added in 16f8c3d3. This caused
redeclaration errors preventing the project from building.
Changes:
- Remove duplicate SyncMode, ConflictStrategy, Sovereignty constants
from config.go (keep typed versions in sync.go)
- Remove duplicate GetSyncMode, GetConflictStrategy, GetSovereignty
functions from config.go (keep sync.go versions with warnings)
- Update SyncConfig, ConflictConfig, FederationConfig structs to use
typed fields instead of string
- Add IsSyncModeValid, IsConflictStrategyValid, IsSovereigntyValid
wrapper functions that use sync.go's validation maps
- Update cmd/bd/sync.go to use typed ConflictStrategy parameter
- Update tests to work with typed constants
* fix(dolt): handle Merge return values in concurrent test
* fix(test): add --repo flag to show_test.go to bypass auto-routing
The tests were failing because the create command was routing issues
to ~/.beads-planning instead of the test's temp directory. Adding
--repo . overrides auto-routing and creates issues in the test dir.
