fix(daemon): prevent zombie state after database file replacement (#1213)

fix(daemon): prevent zombie state after database file replacement

Adds checkFreshness() to health check paths (GetMetadata, GetConfig, GetAllConfig) and refactors reconnect() to validate new connection before closing old.

PR-URL: https://github.com/steveyegge/beads/pull/1213
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

docs/DAEMON.md

@@ -554,6 +554,65 @@ bd daemons logs . -n 200 | grep -i "memory\|leak\|oom"
 - With watcher: ~35MB
 - Per issue: ~500 bytes (10K issues = ~5MB)
 
+## Troubleshooting: "sql: database is closed" Errors
+
+If you see repeated "sql: database is closed" errors in daemon logs:
+
+### Symptoms
+
+- Health checks fail with "sql: database is closed"
+- Daemon appears running (`bd info` shows PID) but commands fail
+- Error persists for extended periods (30+ minutes)
+
+### Cause
+
+Database file was replaced externally (e.g., by `git pull`, `git merge`, or manual operation), and automatic reconnection failed or wasn't triggered.
+
+### Diagnosis
+
+```bash
+# 1. Check if database file was replaced
+ls -li .beads/beads.db
+
+# 2. Enable debug logging
+BD_DEBUG_FRESHNESS=1 bd daemon restart
+
+# 3. Check if daemon has file descriptors to deleted files
+lsof -p $(pgrep -f "bd.*daemon") | grep beads.db
+```
+
+### Solutions
+
+**Immediate fix:**
+```bash
+# Restart daemon
+bd daemon restart
+```
+
+**Enable debug logging** (for investigation):
+```bash
+# Start daemon with freshness debugging
+BD_DEBUG_FRESHNESS=1 bd daemon start --foreground
+
+# Check daemon logs
+bd daemons logs . -n 100 | grep freshness
+```
+
+**Prevention:**
+- Daemon automatically detects file replacement and reconnects (v0.48+)
+- If issue persists, check `.beads/daemon.log` for reconnection errors
+- Report persistent issues with debug logs
+
+### Common Causes
+
+1. **Git operations** (pull, merge, rebase) that replace the database file
+2. **Manual database file replacement** during development
+3. **File system issues** (network file systems, WSL2)
+
+### Technical Details
+
+The daemon monitors database file metadata (inode, mtime) and automatically reconnects when the file is replaced. Health checks call `GetMetadata()` which triggers freshness checking. If reconnection fails, the old connection remains usable until a valid database is restored.
+
 ## Multi-Workspace Best Practices
 
 ### When managing multiple projects:

docs/TROUBLESHOOTING.md

@@ -4,6 +4,7 @@ Common issues and solutions for bd users.
 
 ## Table of Contents
 
+- [Debug Environment Variables](#debug-environment-variables)
 - [Installation Issues](#installation-issues)
 - [Antivirus False Positives](#antivirus-false-positives)
 - [Database Issues](#database-issues)
@@ -13,6 +14,118 @@ Common issues and solutions for bd users.
 - [Agent-Specific Issues](#agent-specific-issues)
 - [Platform-Specific Issues](#platform-specific-issues)
 
+## Debug Environment Variables
+
+bd supports several environment variables for debugging specific subsystems. Enable these when troubleshooting issues or when requested by maintainers.
+
+### Available Debug Variables
+
+| Variable | Purpose | Output Location | Usage |
+|----------|---------|----------------|-------|
+| `BD_DEBUG` | General debug logging | stderr | Set to any value to enable |
+| `BD_DEBUG_RPC` | RPC communication between CLI and daemon | stderr | Set to `1` or `true` |
+| `BD_DEBUG_SYNC` | Sync and import timestamp protection | stderr | Set to any value to enable |
+| `BD_DEBUG_ROUTING` | Issue routing and multi-repo resolution | stderr | Set to any value to enable |
+| `BD_DEBUG_FRESHNESS` | Database file replacement detection | daemon logs | Set to any value to enable |
+
+### Usage Examples
+
+**General debugging:**
+```bash
+# Enable all general debug logging
+export BD_DEBUG=1
+bd ready
+```
+
+**RPC communication issues:**
+```bash
+# Debug daemon communication
+export BD_DEBUG_RPC=1
+bd list
+
+# Example output:
+# [RPC DEBUG] Connecting to daemon at .beads/bd.sock
+# [RPC DEBUG] Sent request: list (correlation_id=abc123)
+# [RPC DEBUG] Received response: 200 OK
+```
+
+**Sync conflicts:**
+```bash
+# Debug timestamp protection during sync
+export BD_DEBUG_SYNC=1
+bd sync
+
+# Example output:
+# [debug] Protected bd-123: local=2024-01-20T10:00:00Z >= incoming=2024-01-20T09:55:00Z
+```
+
+**Routing issues:**
+```bash
+# Debug issue routing in multi-repo setups
+export BD_DEBUG_ROUTING=1
+bd create "Test issue" --rig=planning
+
+# Example output:
+# [routing] Rig "planning" -> prefix plan, path /path/to/planning-repo (townRoot=/path/to/town)
+# [routing] ID plan-123 matched prefix plan -> /path/to/planning-repo/beads
+```
+
+**Database reconnection issues:**
+```bash
+# Debug database file replacement detection
+export BD_DEBUG_FRESHNESS=1
+bd daemon start --foreground
+
+# Example output:
+# [freshness] FreshnessChecker: inode changed 27548143 -> 7945906
+# [freshness] FreshnessChecker: triggering reconnection
+# [freshness] Database file replaced, reconnection triggered
+
+# Or check daemon logs
+BD_DEBUG_FRESHNESS=1 bd daemon restart
+bd daemons logs . -n 100 | grep freshness
+```
+
+**Multiple debug flags:**
+```bash
+# Enable multiple subsystems
+export BD_DEBUG=1
+export BD_DEBUG_RPC=1
+export BD_DEBUG_FRESHNESS=1
+bd daemon start --foreground
+```
+
+### Tips
+
+- **Disable after debugging**: Debug logging can be verbose. Disable by unsetting the variable:
+  ```bash
+  unset BD_DEBUG
+  unset BD_DEBUG_RPC
+  # etc.
+  ```
+
+- **Capture debug output**: Redirect stderr to a file for analysis:
+  ```bash
+  BD_DEBUG=1 bd sync 2> debug.log
+  ```
+
+- **Daemon logs**: `BD_DEBUG_FRESHNESS` output goes to daemon logs, not stderr:
+  ```bash
+  # View daemon logs
+  bd daemons logs . -n 200
+
+  # Or directly:
+  tail -f .beads/daemon.log
+  ```
+
+- **When filing bug reports**: Include relevant debug output to help maintainers diagnose issues faster.
+
+### Related Documentation
+
+- [DAEMON.md](DAEMON.md) - Daemon management and troubleshooting
+- [SYNC.md](SYNC.md) - Git sync behavior and conflict resolution
+- [ROUTING.md](ROUTING.md) - Multi-repo routing configuration
+
 ## Installation Issues
 
 ### `bd: command not found`