docs(dolt): add comprehensive DOLT.md guide (hq-c5d30a)
Add complete documentation for Dolt backend covering: - Migration from SQLite to Dolt - Contributor onboarding (bootstrap on clone) - Troubleshooting common issues - Recovery procedures - Server mode setup and management - Git hooks integration - Configuration reference Also improve server mode error handling to suggest gt dolt start when connection is refused. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
beads/crew/emma
committed by
Steve Yegge
Steve Yegge
parent
f845bb1be3
commit
825e1952c0
393
docs/DOLT.md
Normal file
393
docs/DOLT.md
Normal file
@@ -0,0 +1,393 @@
|
||||
# Dolt Backend for Beads
|
||||
|
||||
Beads supports Dolt as an alternative storage backend to SQLite. Dolt provides version-controlled SQL database capabilities, enabling powerful workflows for multi-agent environments and team collaboration.
|
||||
|
||||
## Why Use Dolt?
|
||||
|
||||
| Feature | SQLite | Dolt |
|
||||
|---------|--------|------|
|
||||
| Version control | Via JSONL export | Native (cell-level) |
|
||||
| Multi-writer | Single process | Server mode supported |
|
||||
| Merge conflicts | Line-based JSONL | Cell-level 3-way merge |
|
||||
| History | Git commits | Dolt commits + Git |
|
||||
| Branching | Via Git branches | Native Dolt branches |
|
||||
|
||||
**Recommended for:**
|
||||
- Multi-agent environments (Gas Town)
|
||||
- Teams wanting database-level version control
|
||||
- Projects needing cell-level conflict resolution
|
||||
|
||||
**Stick with SQLite for:**
|
||||
- Simple single-user setups
|
||||
- Maximum compatibility
|
||||
- Minimal dependencies
|
||||
|
||||
## Getting Started
|
||||
|
||||
### New Project with Dolt
|
||||
|
||||
```bash
|
||||
# Embedded mode (single writer)
|
||||
bd init --backend dolt
|
||||
|
||||
# Server mode (multi-writer)
|
||||
gt dolt start # Start the Dolt server
|
||||
bd init --backend dolt --server # Initialize with server mode
|
||||
```
|
||||
|
||||
### Migrate Existing Project to Dolt
|
||||
|
||||
```bash
|
||||
# Preview the migration
|
||||
bd migrate --to-dolt --dry-run
|
||||
|
||||
# Run the migration
|
||||
bd migrate --to-dolt
|
||||
|
||||
# Optionally clean up SQLite files
|
||||
bd migrate --to-dolt --cleanup
|
||||
```
|
||||
|
||||
Migration creates backups automatically. Your original SQLite database is preserved as `beads.backup-pre-dolt-*.db`.
|
||||
|
||||
### Migrate Back to SQLite (Escape Hatch)
|
||||
|
||||
If you need to revert:
|
||||
|
||||
```bash
|
||||
bd migrate --to-sqlite
|
||||
```
|
||||
|
||||
## Modes of Operation
|
||||
|
||||
### Embedded Mode (Default)
|
||||
|
||||
Single-process access to the Dolt database. Good for development and single-agent use.
|
||||
|
||||
```yaml
|
||||
# .beads/config.yaml (or auto-detected)
|
||||
database: dolt
|
||||
```
|
||||
|
||||
Characteristics:
|
||||
- No server process needed
|
||||
- Single writer at a time
|
||||
- Daemon mode disabled (direct access only)
|
||||
|
||||
### Server Mode (Multi-Writer)
|
||||
|
||||
Connects to a running `dolt sql-server` for multi-client access.
|
||||
|
||||
```bash
|
||||
# Start the server (Gas Town)
|
||||
gt dolt start
|
||||
|
||||
# Or manually
|
||||
cd ~/.dolt-data/beads && dolt sql-server --port 3307
|
||||
```
|
||||
|
||||
```yaml
|
||||
# .beads/config.yaml
|
||||
database: dolt
|
||||
dolt:
|
||||
mode: server
|
||||
host: 127.0.0.1
|
||||
port: 3307
|
||||
user: root
|
||||
```
|
||||
|
||||
Server mode is required for:
|
||||
- Multiple agents writing simultaneously
|
||||
- Gas Town multi-rig setups
|
||||
- Federation with remote peers
|
||||
|
||||
## Contributor Onboarding (Clone Bootstrap)
|
||||
|
||||
When someone clones a repository that uses Dolt backend:
|
||||
|
||||
1. They see the `issues.jsonl` file (committed to git)
|
||||
2. On first `bd` command (e.g., `bd list`), bootstrap runs automatically
|
||||
3. JSONL is imported into a fresh Dolt database
|
||||
4. Work continues normally
|
||||
|
||||
**No manual steps required.** The bootstrap:
|
||||
- Detects fresh clone (JSONL exists, Dolt doesn't)
|
||||
- Acquires a lock to prevent race conditions
|
||||
- Imports issues, routes, interactions, labels, dependencies
|
||||
- Creates initial Dolt commit "Bootstrap from JSONL"
|
||||
|
||||
### Verifying Bootstrap Worked
|
||||
|
||||
```bash
|
||||
bd list # Should show issues
|
||||
bd vc log # Should show "Bootstrap from JSONL" commit
|
||||
```
|
||||
|
||||
## Git Hooks Integration
|
||||
|
||||
Dolt uses specialized hooks for JSONL synchronization:
|
||||
|
||||
| Hook | Purpose |
|
||||
|------|---------|
|
||||
| pre-commit | Export Dolt changes to JSONL, stage for commit |
|
||||
| post-merge | Import pulled JSONL changes into Dolt |
|
||||
|
||||
### Installing Hooks
|
||||
|
||||
```bash
|
||||
# Recommended for Dolt projects
|
||||
bd hooks install --beads
|
||||
|
||||
# Or shared across team
|
||||
bd hooks install --shared
|
||||
```
|
||||
|
||||
### How Hooks Work
|
||||
|
||||
**Pre-commit (export):**
|
||||
1. Checks if Dolt has changes since last export
|
||||
2. Exports database to `issues.jsonl`
|
||||
3. Stages JSONL file for git commit
|
||||
4. Tracks export state per-worktree
|
||||
|
||||
**Post-merge (import):**
|
||||
1. Creates temporary Dolt branch
|
||||
2. Imports JSONL to that branch
|
||||
3. Merges using Dolt's cell-level 3-way merge
|
||||
4. Deletes temporary branch
|
||||
|
||||
The branch-then-merge pattern provides better conflict resolution than line-based JSONL merging.
|
||||
|
||||
### Verifying Hooks
|
||||
|
||||
```bash
|
||||
bd hooks list # Shows installed hooks
|
||||
bd doctor # Checks hook health
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Not Running
|
||||
|
||||
**Symptom:** Connection refused errors when using server mode.
|
||||
|
||||
```
|
||||
failed to create database: dial tcp 127.0.0.1:3307: connect: connection refused
|
||||
```
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
gt dolt start # Gas Town command
|
||||
# Or
|
||||
gt dolt status # Check if running
|
||||
```
|
||||
|
||||
### Bootstrap Not Running
|
||||
|
||||
**Symptom:** `bd list` shows nothing on fresh clone.
|
||||
|
||||
**Check:**
|
||||
```bash
|
||||
ls .beads/issues.jsonl # Should exist
|
||||
ls .beads/dolt/ # Should NOT exist (pre-bootstrap)
|
||||
BD_DEBUG=1 bd list # See bootstrap output
|
||||
```
|
||||
|
||||
**Force bootstrap:**
|
||||
```bash
|
||||
rm -rf .beads/dolt # Remove broken state
|
||||
bd list # Re-triggers bootstrap
|
||||
```
|
||||
|
||||
### Database Corruption
|
||||
|
||||
**Symptom:** Queries fail, inconsistent data.
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
bd doctor # Basic checks
|
||||
bd doctor --deep # Full validation
|
||||
bd doctor --server # Server mode checks (if applicable)
|
||||
```
|
||||
|
||||
**Recovery options:**
|
||||
|
||||
1. **Repair what's fixable:**
|
||||
```bash
|
||||
bd doctor --fix
|
||||
```
|
||||
|
||||
2. **Nuclear option (rebuild from JSONL):**
|
||||
```bash
|
||||
rm -rf .beads/dolt
|
||||
bd sync # Rebuilds from JSONL
|
||||
```
|
||||
|
||||
3. **Restore from backup:**
|
||||
```bash
|
||||
# If you have a pre-migration backup
|
||||
ls .beads/*.backup-*.db
|
||||
```
|
||||
|
||||
### Hooks Not Firing
|
||||
|
||||
**Symptom:** JSONL not updating on commit, or Dolt not updating on pull.
|
||||
|
||||
**Check:**
|
||||
```bash
|
||||
bd hooks list # See what's installed
|
||||
git config core.hooksPath # May override .git/hooks
|
||||
bd doctor # Checks hook health
|
||||
```
|
||||
|
||||
**Reinstall:**
|
||||
```bash
|
||||
bd hooks install --beads --force
|
||||
```
|
||||
|
||||
### Migration Failed Halfway
|
||||
|
||||
**Symptom:** Both SQLite and Dolt exist, unclear state.
|
||||
|
||||
**Recovery:**
|
||||
```bash
|
||||
# Check what exists
|
||||
ls .beads/*.db .beads/dolt/
|
||||
|
||||
# If Dolt looks incomplete, restart migration
|
||||
rm -rf .beads/dolt
|
||||
bd migrate --to-dolt
|
||||
|
||||
# If you want to abandon migration
|
||||
rm -rf .beads/dolt
|
||||
# SQLite remains as primary
|
||||
```
|
||||
|
||||
### Lock Contention (Embedded Mode)
|
||||
|
||||
**Symptom:** "database is locked" errors.
|
||||
|
||||
Embedded mode is single-writer. If you need concurrent access:
|
||||
|
||||
```bash
|
||||
# Switch to server mode
|
||||
gt dolt start
|
||||
bd config set dolt.mode server
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
```yaml
|
||||
# .beads/config.yaml
|
||||
|
||||
# Database backend
|
||||
database: dolt # sqlite | dolt
|
||||
|
||||
# Dolt-specific settings
|
||||
dolt:
|
||||
# Auto-commit Dolt history after writes (default: on)
|
||||
auto-commit: on # on | off
|
||||
|
||||
# Server mode settings (when mode: server)
|
||||
mode: embedded # embedded | server
|
||||
host: 127.0.0.1
|
||||
port: 3307
|
||||
user: root
|
||||
# Password via BEADS_DOLT_PASSWORD env var
|
||||
|
||||
# Sync mode (how JSONL and database stay in sync)
|
||||
sync:
|
||||
mode: git-portable # git-portable | dolt-native | belt-and-suspenders
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `BD_BACKEND` | Override backend (sqlite/dolt) |
|
||||
| `BD_DOLT_MODE` | Override mode (embedded/server) |
|
||||
| `BEADS_DOLT_PASSWORD` | Server mode password |
|
||||
| `BD_DOLT_AUTO_COMMIT` | Override auto-commit setting |
|
||||
|
||||
## Dolt Version Control
|
||||
|
||||
Dolt maintains its own version history, separate from Git:
|
||||
|
||||
```bash
|
||||
# View Dolt commit history
|
||||
bd vc log
|
||||
|
||||
# Show diff between Dolt commits
|
||||
bd vc diff HEAD~1 HEAD
|
||||
|
||||
# Create manual checkpoint
|
||||
bd vc commit -m "Checkpoint before refactor"
|
||||
```
|
||||
|
||||
### Auto-Commit Behavior
|
||||
|
||||
By default, each `bd` write command creates a Dolt commit:
|
||||
|
||||
```bash
|
||||
bd create "New issue" # Creates issue + Dolt commit
|
||||
```
|
||||
|
||||
Disable for batch operations:
|
||||
|
||||
```bash
|
||||
bd --dolt-auto-commit off create "Issue 1"
|
||||
bd --dolt-auto-commit off create "Issue 2"
|
||||
bd vc commit -m "Batch: created issues"
|
||||
```
|
||||
|
||||
## Server Management (Gas Town)
|
||||
|
||||
Gas Town provides integrated Dolt server management:
|
||||
|
||||
```bash
|
||||
gt dolt start # Start server (background)
|
||||
gt dolt stop # Stop server
|
||||
gt dolt status # Show server status
|
||||
gt dolt logs # View server logs
|
||||
gt dolt sql # Open SQL shell
|
||||
```
|
||||
|
||||
Server runs on port 3307 (avoids MySQL conflict on 3306).
|
||||
|
||||
### Data Location
|
||||
|
||||
```
|
||||
~/.dolt-data/
|
||||
├── beads/ # HQ database
|
||||
├── beads_rig/ # Beads rig database
|
||||
└── gastown/ # Gas Town database
|
||||
```
|
||||
|
||||
## Migration Cleanup
|
||||
|
||||
After successful migration, you may have backup files:
|
||||
|
||||
```
|
||||
.beads/beads.backup-pre-dolt-20260122-213600.db
|
||||
.beads/sqlite.backup-pre-dolt-20260123-192812.db
|
||||
```
|
||||
|
||||
These are safe to delete once you've verified Dolt is working:
|
||||
|
||||
```bash
|
||||
# Verify Dolt works
|
||||
bd list
|
||||
bd doctor
|
||||
|
||||
# Then clean up (after appropriate waiting period)
|
||||
rm .beads/*.backup-*.db
|
||||
```
|
||||
|
||||
**Recommendation:** Keep backups for at least a week before deleting.
|
||||
|
||||
## See Also
|
||||
|
||||
- [CONFIG.md](CONFIG.md) - Full configuration reference
|
||||
- [GIT_INTEGRATION.md](GIT_INTEGRATION.md) - Git hooks and sync workflows
|
||||
- [TROUBLESHOOTING.md](TROUBLESHOOTING.md) - General troubleshooting
|
||||
- [SYNC.md](SYNC.md) - Sync modes and strategies
|
||||
Reference in New Issue
Block a user