Commit Graph

12 Commits

Author SHA1 Message Date
Charles P. Cross
03f5afb605 feat(daemon): add auto_pull config parameter for periodic remote sync
Add --auto-pull flag to control whether the daemon periodically pulls from
remote to check for updates from other clones.

Configuration precedence:
1. --auto-pull CLI flag (highest)
2. BEADS_AUTO_PULL environment variable
3. daemon.auto_pull in database config
4. Default: true when sync.branch is configured

When auto_pull is enabled, the daemon creates a remoteSyncTicker that
periodically calls doAutoImport() to pull remote changes. When disabled,
users must manually run 'git pull' to sync remote changes.

Changes:
- cmd/bd/daemon.go: Add --auto-pull flag and config reading logic
- cmd/bd/daemon_event_loop.go: Gate remoteSyncTicker on autoPull parameter
- cmd/bd/daemon_lifecycle.go: Add auto-pull to status output and spawn args
- internal/rpc/protocol.go: Add AutoPull field to StatusResponse
- internal/rpc/server_core.go: Add autoPull to Server struct and SetConfig
- internal/rpc/server_routing_validation_diagnostics.go: Include in status
- Tests updated to pass autoPull parameter

Closes #TBD
2025-12-22 18:47:18 -05:00
Charles P. Cross
737e65afbd fix(daemon): add periodic remote sync to event-driven mode (#698)
* fix(daemon): add periodic remote sync to event-driven mode

The event-driven daemon mode only triggered imports when the local JSONL
file changed (via file watcher) or when the fallback ticker fired (only
if watcher failed). This meant the daemon wouldn't see updates pushed
by other clones until something triggered a local file change.

Bug scenario:
1. Clone A creates an issue and daemon pushes to sync branch
2. Clone B's daemon only watched local file changes
3. Clone B would not see the new issue until something triggered local change
4. With this fix: Clone B's daemon periodically calls doAutoImport

This fix adds a 30-second periodic remote sync ticker that calls
doAutoImport(), which includes syncBranchPull() to fetch and import
updates from the remote sync branch.

This is essential for multi-clone workflows where:
- Clone A creates an issue and daemon pushes to sync branch
- Clone B's daemon needs to periodically pull to see the new issue
- Without periodic sync, Clone B would only see updates if its local
  JSONL file happened to change

The 30-second interval balances responsiveness with network overhead.

Adds integration test TestEventDrivenLoop_PeriodicRemoteSync that
verifies the event-driven loop starts with periodic sync support.

* feat(daemon): add configurable interval for periodic remote sync

- Add BEADS_REMOTE_SYNC_INTERVAL environment variable to configure
  the interval for periodic remote sync (default: 30s)
- Add getRemoteSyncInterval() function to parse the env var
- Minimum interval is 5s to prevent excessive load
- Setting to 0 disables periodic sync (not recommended)
- Add comprehensive integration tests for the configuration

Valid duration formats:
- "30s" (30 seconds)
- "1m" (1 minute)
- "5m" (5 minutes)

Tests added:
- TestEventDrivenLoop_HasRemoteSyncTicker
- TestGetRemoteSyncInterval_Default
- TestGetRemoteSyncInterval_CustomValue
- TestGetRemoteSyncInterval_MinimumEnforced
- TestGetRemoteSyncInterval_InvalidValue
- TestGetRemoteSyncInterval_Zero
- TestSyncBranchPull_FetchesRemoteUpdates

* fix: resolve all golangci-lint errors (cherry-pick from fix/linting-errors)

Cherry-picked linting fixes to ensure CI passes.

* feat(daemon): add config.yaml support for remote-sync-interval

- Add remote-sync-interval to .beads/config.yaml as alternative to
  BEADS_REMOTE_SYNC_INTERVAL environment variable
- Environment variable takes precedence over config.yaml (follows
  existing pattern for flush-debounce)
- Add config binding in internal/config/config.go
- Update getRemoteSyncInterval() to use config.GetDuration()
- Add doctor validation for remote-sync-interval in config.yaml

Configuration sources (in order of precedence):
1. BEADS_REMOTE_SYNC_INTERVAL environment variable
2. remote-sync-interval in .beads/config.yaml
3. DefaultRemoteSyncInterval (30s)

Example config.yaml:
  remote-sync-interval: "1m"

---------

Co-authored-by: Charles P. Cross <cpdata@users.noreply.github.com>
2025-12-22 14:15:33 -08:00
Steve Yegge
f59f8c20d0 refactor: rename last_import_hash to jsonl_content_hash (bd-39o)
The metadata key 'last_import_hash' was misleading because it's updated on
both import AND export. Renamed to 'jsonl_content_hash' which more accurately
describes its purpose - tracking the content hash of the JSONL file.

Added migration support: read operations try new key first, then fall back
to old key for backwards compatibility with existing databases.

Files modified:
- cmd/bd/integrity.go: Update key name with migration support
- cmd/bd/import.go: Update key name
- cmd/bd/sync.go: Update key name
- cmd/bd/autoflush.go: Update key name with migration support
- cmd/bd/daemon_sync.go: Update key name
- cmd/bd/daemon_event_loop.go: Update key name with migration support
- internal/autoimport/autoimport.go: Update key name with migration support
- Updated all related tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 23:14:12 -08:00
Steve Yegge
39909c1f80 feat: implement health checks in daemon event loop (bd-gqo)
Add health checks to checkDaemonHealth() function:
- Database integrity check using PRAGMA quick_check(1)
- Disk space check with 100MB warning threshold (platform-specific)
- Memory usage check with 500MB heap warning threshold

Platform-specific disk space implementations:
- Unix: uses unix.Statfs
- Windows: uses windows.GetDiskFreeSpaceEx
- WASM: returns unsupported (false)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 23:10:44 -08:00
Steve Yegge
58fe00057c feat: Complete GH #353 follow-up phases (bd-9nw, bd-u3t, bd-e0o)
Implements all three follow-up phases for sandbox environment support:

**Phase 1 (bd-9nw): Documentation** 
- Comprehensive sandbox troubleshooting section in TROUBLESHOOTING.md
  - Detailed symptoms, root causes, and escape hatches
  - Step-by-step troubleshooting workflow
  - Comparison table for --sandbox, --force, and --allow-stale flags
- Global flags section added to CLI_REFERENCE.md
  - Documents --sandbox, --allow-stale, and --force flags
  - Usage examples and when to use each flag
- GitHub issue #353 comment with immediate workarounds

**Phase 2 (bd-u3t): Sandbox Auto-Detection** 
- Automatic sandbox detection using syscall.Kill permission checks
  - cmd/bd/sandbox_unix.go: Unix/Linux/macOS implementation
  - cmd/bd/sandbox_windows.go: Windows stub (conservative approach)
  - cmd/bd/sandbox_test.go: Comprehensive test coverage
- Auto-enables sandbox mode when detected
  - Shows: "ℹ️  Sandbox detected, using direct mode"
  - Respects explicit --sandbox or --no-daemon flags
- Updated documentation to reflect auto-detection (v0.21.1+)

**Phase 3 (bd-e0o): Enhanced Daemon Robustness** 
- Permission-aware process checks in cmd/bd/daemon_unix.go
  - Correctly handles EPERM (operation not permitted) from syscall.Kill
  - Treats EPERM as "process exists but not signable" = running
  - Prevents false negatives in sandboxed environments
- Metadata health check in cmd/bd/daemon_event_loop.go
  - Periodic verification that metadata is accessible
  - Helps detect external import operations (bd import --force)
  - Non-fatal logging for diagnostics
- Comprehensive test suite in cmd/bd/daemon_unix_test.go
  - Self-check, init process, nonexistent process, parent process tests

**Impact:**
- Codex users: No manual intervention needed, auto-detected
- Stuck states: Three escape hatches (--sandbox, --force, --allow-stale)
- Daemon robustness: Handles permission-restricted environments gracefully
- All three follow-up issues (bd-9nw, bd-u3t, bd-e0o) closed

**Files changed:**
- cmd/bd/main.go: Auto-detection logic in PersistentPreRun
- cmd/bd/sandbox_unix.go: Unix sandbox detection (new)
- cmd/bd/sandbox_windows.go: Windows sandbox detection stub (new)
- cmd/bd/sandbox_test.go: Sandbox detection tests (new)
- cmd/bd/daemon_unix.go: Permission-aware isProcessRunning()
- cmd/bd/daemon_unix_test.go: Process check tests (new)
- cmd/bd/daemon_event_loop.go: Metadata health check
- docs/TROUBLESHOOTING.md: Comprehensive sandbox section
- docs/CLI_REFERENCE.md: Global flags documentation

Closes bd-9nw, bd-u3t, bd-e0o
Related: GH #353

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 19:32:45 -05:00
Steve Yegge
65ba2c9a19 feat: Add issue IDs to TODO comments
Updated TODO comments with proper beads issue IDs:
- bd-hdt: Implement auto-merge functionality
- bd-gqo: Implement daemon health checks
- bd-r46: Support --reason flag in daemon mode for reopen

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 19:01:42 -05:00
Steve Yegge
7fa0c93195 Fix daemon orphaning: track parent PID and exit when parent dies
- Add ParentPID field to DaemonLockInfo struct
- Daemon monitors parent process every 10 seconds
- Gracefully exits when parent process dies
- Prevents accumulation of orphaned daemons from dead sessions
- Fixes race conditions from multiple daemons on same database

Closes bd-zpnq
2025-11-07 18:57:43 -08:00
Steve Yegge
2b086951c4 fix: Address all errcheck and misspell linter errors 2025-11-01 23:56:03 -07:00
Steve Yegge
349b892123 Harden event-driven daemon for production
Three critical fixes to make event-driven mode production-ready:

1. Skip redundant imports: Check JSONL mtime vs DB mtime to avoid
   self-triggered import loops after export writes JSONL

2. Add server.Stop() in serverErrChan case: Ensures clean RPC
   server shutdown on errors

3. Fallback ticker (60s): When file watcher unavailable (e.g., network
   filesystems), fall back to periodic polling to detect remote changes

These minimal fixes address Oracle's concerns without over-engineering.
Event-driven mode is now safe for default.

Amp-Thread-ID: https://ampcode.com/threads/T-a9a67394-37ca-4b79-aa23-c5c011f9c0cd
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 20:18:05 -07:00
Steve Yegge
e2bb4311f1 Improve mutation channel robustness
- Add event type constants (MutationCreate, MutationUpdate, MutationDelete, MutationComment)
- Make buffer size configurable via BEADS_MUTATION_BUFFER (default 512, up from 100)
- Add defensive closed-channel handling in event loop consumer
- Add 1s dropped-events ticker (down from 60s) for faster recovery
- Verify mutationChan is never closed (prevents panic)

Oracle review findings addressed:
- Eliminates panic risk from send-on-closed-channel
- Reduces worst-case recovery latency from 60s to 1s
- Increases buffer capacity for better burst handling
- Type-safe event constants prevent string typos

Related: bd-36320a04, bd-1f4086c5
2025-10-31 19:12:02 -07:00
Steve Yegge
fea86f9b31 Implement event-driven daemon improvements for bd-85
- Add mutation events for label/dep/comment operations
- Create separate export-only and import-only functions
- Add dropped events counter with safety net export
- Complete bd-80 mutation channel implementation

Event-driven mode now:
- Emits mutation events for ALL write operations (not just create/update/close)
- Uses createExportFunc() for mutations (export+commit/push only, no pull)
- Uses createAutoImportFunc() for file changes (pull+import only, no export)
- Tracks dropped events and triggers safety export every 60s if any dropped
- Achieves <500ms latency target by avoiding full sync on each trigger

Behind BEADS_DAEMON_MODE=events flag (poll is still default)
2025-10-29 11:22:29 -07:00
Steve Yegge
781e300d33 Add event-driven daemon architecture (Phase 1 foundation)
- Add fsnotify dependency for file watching
- Create daemon_debouncer.go: batch rapid events (500ms window)
- Create daemon_watcher.go: monitor JSONL and git refs changes
- Create daemon_event_loop.go: event-driven sync loop
- Add mutation channel to RPC server (create/update/close events)
- Add BEADS_DAEMON_MODE env var (poll/events, default: poll)

Phase 1 implementation: opt-in via BEADS_DAEMON_MODE=events
Target: <500ms latency (vs 5000ms), ~60% CPU reduction

Related: bd-49 (epic), bd-50, bd-51, bd-53, bd-54, bd-55, bd-56
Amp-Thread-ID: https://ampcode.com/threads/T-35a3d0d7-4e19-421d-8392-63755035036e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-28 13:12:37 -07:00