Commit Graph

14 Commits

Author SHA1 Message Date
Steve Yegge
b409870ce5 Fix linter issues and test failures
- Fix spelling: cancelled → canceled, cancelling → canceling
- Remove unused error return from performFlush() function
- Check cmd.Help() error return in search.go
- Fix file permissions in gitignore.go (0644 → 0600)
- Update tests to match performFlush signature change
- Remove unused os import from flush_manager.go
2025-11-22 00:11:45 -08:00
Steve Yegge
87ee3a674e Improve bd doctor JSONL checks to focus on real problems
Previously bd doctor warned about using beads.jsonl vs issues.jsonl, but
users should be free to configure any name they want. The real problems are:
1. Having multiple JSONL files (sync/merge conflicts)
2. Configuration not matching reality

Changes:
- Rewrote CheckLegacyJSONLFilename to scan for ALL .jsonl files
- Now filters out merge artifacts (backup, .orig, .bak, etc.)
- Warns only when multiple real JSONL files exist
- Added CheckDatabaseConfig to detect when configured paths do not match
  what actually exists on disk
- Updated tests to verify backup files are ignored
- Added test cases for custom JSONL filenames

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:44:26 -08:00
Steve Yegge
c4c5c8063a Fix: Change default JSONL filename from beads.jsonl to issues.jsonl
The canonical beads database name is issues.jsonl. Tens of thousands of users
have issues.jsonl, and beads.jsonl was only used by the Beads project itself
due to git history pollution.

Changes:
- Updated bd doctor to warn about beads.jsonl instead of issues.jsonl
- Changed default config from beads.jsonl to issues.jsonl
- Reversed precedence in checkGitForIssues to prefer issues.jsonl
- Updated git merge driver config to use issues.jsonl
- Updated all tests to expect issues.jsonl as the default

issues.jsonl is now the canonical default; beads.jsonl is legacy

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:34:22 -08:00
Steve Yegge
6e702d3ecd Merge branch 'main' of https://github.com/steveyegge/beads 2025-11-21 20:40:23 -05:00
Steve Yegge
ddd209e26a fix: Correct git merge driver placeholders from %L/%R to %A/%B
Git merge drivers only support three placeholders:
- %O (ancestor/base)
- %A (current version)
- %B (other branch's version)

The code was incorrectly using %L and %R, which don't exist in git,
causing them to be passed through literally and breaking JSONL merges.

Changes:
- Fixed merge driver config in init.go, merge.go, README.md, docs
- Added detection in bd doctor with clear error messages
- Added auto-fix in bd doctor --fix
- Added proactive warning in bd sync before git pull
- Added reactive error detection after merge failures
- Updated all tests to use correct placeholders

Now users get helpful guidance at every step:
1. bd doctor detects the issue
2. bd doctor --fix auto-corrects it
3. bd sync warns before pulling if misconfigured
4. Error messages suggest bd doctor --fix when merge fails

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 20:40:18 -05:00
Steve Yegge
8efab3df87 chore: Move design/audit docs from cmd/bd to docs/
Relocate documentation files to centralize all .md files in docs/ directory
(except those with historical precedent in specific locations like commands/,
examples/, integrations/, etc.).

Files moved:
- cmd/bd/doctor/claude.md -> docs/CLAUDE_INTEGRATION.md
- cmd/bd/MAIN_TEST_REFACTOR_NOTES.md -> docs/MAIN_TEST_REFACTOR_NOTES.md
- cmd/bd/TEST_SUITE_AUDIT.md -> docs/TEST_SUITE_AUDIT.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 20:31:47 -05:00
Steve Yegge
e447acce9e Merge branch 'main' of https://github.com/steveyegge/beads 2025-11-20 19:37:01 -05:00
Steve Yegge
f01326832f Enhance bd doctor with bd prime migration recommendations (bd-0fvq)
Add comprehensive detection and migration guidance for old beads integration
patterns. This helps users adopt the more efficient bd prime approach.

Changes:
- Enhanced CheckLegacyBeadsSlashCommands with detailed migration steps
  and token efficiency benefits (99% reduction: ~10.5k → ~50 tokens)
- Added CheckAgentDocumentation to detect missing AGENTS.md/CLAUDE.md
  and suggest bd onboard or bd setup claude
- Enhanced CheckClaude to recommend bd prime hooks for MCP-only setups
  with clear token efficiency messaging
- Added comprehensive tests for all new checks

bd doctor now detects:
1. Old slash command patterns (/beads:*) and recommends bd prime hooks
2. Missing agent documentation and suggests creating it
3. MCP-only setups without hooks and shows token savings potential
4. Provides clear migration paths and benefits for all scenarios

Token efficiency messaging:
- MCP mode: ~50 tokens vs ~10.5k for full scan (99% reduction)
- CLI mode: ~1-2k tokens with automatic context recovery
- Hooks auto-refresh context on SessionStart and PreCompact

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:36:48 -05:00
Steve Yegge
7806937b0a Add bd doctor --fix flag to automatically repair issues (bd-ykd9)
Implements automatic fix capability for bd doctor with user confirmation
and security hardening.

Features:
- Organizes fix implementations under doctor/fix/ directory structure
- Shows all fixable issues and prompts for confirmation (Y/n) before applying
- Provides clear output about what was fixed and any errors encountered
- Re-runs diagnostics after fixes to show updated state
- Each fix is idempotent and safe to run multiple times

Automatic fixes implemented:
- Git hooks (runs bd hooks install)
- Daemon health issues (runs bd daemons killall)
- DB-JSONL sync problems (runs bd sync --import-only)
- File permissions (fixes .beads/ and database permissions)
- Database version mismatches (runs bd migrate)
- Schema compatibility issues (runs bd migrate)
- Gitignore updates (writes canonical template)

Security improvements:
- Prevents command injection by using os.Executable() instead of PATH lookup
- Prevents path traversal attacks with workspace validation
- Fixes race conditions by using cmd.Dir instead of os.Chdir()
- Corrects file permission logic (proper bit masking)
- Validates all operations run in beads workspaces only

Files changed:
- cmd/bd/doctor.go: Enhanced applyFixes() with confirmation and better UX
- cmd/bd/doctor/gitignore.go: Fixed permissions (0600 → 0644)
- cmd/bd/doctor/fix/common.go: Security helpers (getBdBinary, validateBeadsWorkspace)
- cmd/bd/doctor/fix/hooks.go: Git hooks fix
- cmd/bd/doctor/fix/daemon.go: Daemon health fix
- cmd/bd/doctor/fix/sync.go: DB-JSONL sync fix
- cmd/bd/doctor/fix/permissions.go: File permissions fix
- cmd/bd/doctor/fix/migrate.go: Database migration fixes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 19:33:12 -05:00
Codex Agent
bf9b2c83fb Annotate gosec-safe file accesses 2025-11-17 10:12:46 -07:00
Codex Agent
7b63b5a30b Fix CI regressions and stabilize tests 2025-11-17 10:06:35 -07:00
Ryan
690c73fc31 Performance Improvements (#319)
* feat: add performance testing framework foundation

Implements foundation for comprehensive performance testing and user
diagnostics for beads databases at 10K-20K scale.

Components added:
- Fixture generator (internal/testutil/fixtures/) for realistic test data
  * LargeSQLite/XLargeSQLite: 10K/20K issues with epic hierarchies
  * LargeFromJSONL/XLargeFromJSONL: test JSONL import path
  * Realistic cross-linked dependencies, labels, assignees
  * Reproducible with seeded RNG

- User diagnostics (bd doctor --perf) for field performance data
  * Collects platform info (OS, arch, Go/SQLite versions)
  * Measures key operation timings (ready, list, show, search)
  * Generates CPU profiles for bug reports
  * Clean separation in cmd/bd/doctor/perf.go

Test data characteristics:
- 10% epics, 30% features, 60% tasks
- 4-level hierarchies (Epic → Feature → Task → Subtask)
- 20% cross-epic blocking dependencies
- Realistic status/priority/label distributions

Supports bd-l954 (Performance Testing Framework epic)
Closes bd-6ed8, bd-q59i

* perf: optimize GetReadyWork with compound index (20x speedup)

Add compound index on dependencies(depends_on_id, type, issue_id) to
eliminate performance bottleneck in GetReadyWork recursive CTE query.

Performance improvements (10K issue database):
- GetReadyWork: 752ms → 36.6ms (20.5x faster)
- Target: <50ms ✓ ACHIEVED
- 20K database: ~1500ms → 79.4ms (19x faster)

Benchmark infrastructure enhancements:
- Add dataset caching in /tmp/beads-bench-cache/ to avoid regenerating
  10K-20K issues on every benchmark run (first run: ~2min, subsequent: <5s)
- Add progress logging during fixture generation (shows 10%, 20%... completion)
- Add database size logging (17.5 MB for 10K, 35.1 MB for 20K)
- Document rationale for only benchmarking large datasets (>10K issues)
- Add CPU/trace profiling with --profile flag for performance debugging

Schema changes:
- internal/storage/sqlite/schema.go: Add idx_dependencies_depends_on_type_issue

New files:
- internal/storage/sqlite/bench_helpers_test.go: Reusable benchmark setup with caching
- internal/storage/sqlite/sqlite_bench_test.go: Comprehensive benchmarks for critical operations
- Makefile: Convenient benchmark execution (make bench-quick, make bench)

Related:
- Resolves bd-5qim (optimize GetReadyWork performance)
- Builds on bd-6ed8 (fixture generator), bd-q59i (bd doctor --perf)

* perf: add WASM compilation cache to eliminate cold-start overhead

Configure wazero compilation cache for ncruces/go-sqlite3 to avoid
~220ms JIT compilation on every process start.

Cache configuration:
- Location: ~/.cache/beads/wasm/ (platform-specific via os.UserCacheDir)
- Automatic version management: wazero keys entries by its version
- Fallback: in-memory cache if directory creation fails
- No cleanup needed: old versions are harmless (~5-10MB each)

Performance impact:
- First run: ~220ms (populate cache)
- Subsequent runs: ~20ms (load from cache)
- Savings: ~200ms per cold start

Cache invalidation:
- Automatic when wazero version changes (upgrades use new cache dir)
- Manual cleanup: rm -rf ~/.cache/beads/wasm/ (safe to delete anytime)

This complements daemon mode:
- Daemon mode: eliminates startup cost by keeping process alive
- WASM cache: reduces startup cost for one-off commands or daemon restarts

Changes:
- internal/storage/sqlite/sqlite.go: Add init() with cache setup

* refactor: improve maintainability of performance testing code

Extract common patterns and eliminate duplication across benchmarks, fixture generation, and performance diagnostics. Replace magic numbers with explicit configuration to improve readability and make it easier to tune test parameters.

* docs: clarify profiling behavior and add missing documentation

Add explanatory comments for profiling setup to clarify why --profile
forces direct mode (captures actual database operations instead of RPC
overhead) and document the stopCPUProfile function's role in flushing
profile data to disk. Also fix gosec G104 linter warning by explicitly
ignoring Close() error during cleanup.

* fix: prevent bench-quick from running indefinitely

Added //go:build bench tags and skipped timeout-prone benchmarks to
prevent make bench-quick from running for hours.

Changes:
- Add //go:build bench tag to cycle_bench_test.go and compact_bench_test.go
- Skip Dense graph benchmarks (documented to timeout >120s)
- Fix compact benchmark prefix: bd- → bd (validation expects prefix without trailing dash)

Before: make bench-quick ran for 3.5+ hours (12,699s) before manual interrupt
After: make bench-quick completes in ~25 seconds

The Dense graph benchmarks are known to timeout and represent rare edge
cases that don't need optimization for typical workflows.
2025-11-15 12:46:13 -08:00
Steve Yegge
f4a2f87aff Fix #274: Add automatic .beads/.gitignore upgrade (#300)
* Fix #274: Add automatic .beads/.gitignore upgrade

Implements three mechanisms to ensure users get updated gitignore:

1. bd doctor --fix: Manually upgrade gitignore
2. Daemon auto-upgrade: Upgrades on startup if outdated
3. bd init idempotent: Safe to re-run, always updates gitignore

The gitignore template now lives in cmd/bd/doctor/gitignore.go
for consistent updates across all three mechanisms.

Fixes: #274

* Remove test binary

Amp-Thread-ID: https://ampcode.com/threads/T-7042cfcc-ac97-43d7-a40f-3fa1bb4e1c2b
Co-authored-by: Amp <amp@ampcode.com>

* Fix critical issues: remove merge artifact and apply gitignore template

- Remove .beads/beads.left.jsonl (merge artifact that shouldn't be committed)
- Apply new gitignore template to .beads/.gitignore (was missing patterns)

Amp-Thread-ID: https://ampcode.com/threads/T-7042cfcc-ac97-43d7-a40f-3fa1bb4e1c2b
Co-authored-by: Amp <amp@ampcode.com>

* bd sync: 2025-11-12 11:09:30

* Retrigger CI

Amp-Thread-ID: https://ampcode.com/threads/T-8d532264-6d5e-4b68-88e9-e4511851b64a
Co-authored-by: Amp <amp@ampcode.com>

* Fix duplicate DoctorCheck type definition

* Trigger CI after fixing type conflict

Amp-Thread-ID: https://ampcode.com/threads/T-8d532264-6d5e-4b68-88e9-e4511851b64a
Co-authored-by: Amp <amp@ampcode.com>

---------

Co-authored-by: Amp <amp@ampcode.com>
2025-11-12 12:46:27 -08:00
Ryan
f7e80dd80c feat: add context optimization features for AI agents (#297)
* feat: add bd prime and setup commands for AI agent integration

This commit consolidates context optimization features for AI agents:

## New Commands

**bd prime** - AI-optimized workflow context injection
- Outputs ~1-2k tokens of workflow context
- Context-aware: adapts to MCP vs CLI mode
- MCP mode: minimal reminders (~500 tokens)
- CLI mode: full command reference (~1-2k tokens)
- Warns against TodoWrite tool and markdown TODOs
- Designed for SessionStart/PreCompact hooks

**bd setup claude** - Claude Code integration installer
- Installs hooks via JSON configuration (not file scripts)
- Supports --project for project-only installation
- Supports --check to verify installation
- Supports --remove to uninstall hooks
- Idempotent (safe to run multiple times)
- Merges with existing settings

**bd setup cursor** - Cursor IDE integration installer
- Creates .cursor/rules/beads.mdc with workflow rules
- Simplified implementation (just overwrites file)

## bd doctor Enhancements

- New: CheckClaude() verifies Claude Code integration
- Detects plugin, MCP server, and hooks installation
- Provides actionable fix suggestions
- Extracted legacy pattern detection to doctor/legacy.go
- Detects JSONL-only mode and warns about legacy issues.jsonl

## Core Improvements

- FindBeadsDir() utility for cross-platform .beads/ discovery
- Works in JSONL-only mode (no database required)
- Sorted noDbCommands alphabetically (one per line for easy diffs)

## Testing

- Unit tests for setup command hook manipulation
- Tests for idempotency, adding/removing hooks
- All tests passing

## Documentation

- cmd/bd/doctor/claude.md - Documents why beads doesn't use Claude Skills
- commands/prime.md - Slash command for bd prime
- Fixed G304 gosec warnings with nosec comments

## Token Efficiency

The bd prime approach reduces AI context usage dramatically:
- MCP mode: ~500 tokens (vs ~10.5k for full MCP tool scan)
- CLI mode: ~1-2k tokens
- 80-99% reduction in standing context overhead

* fix: resolve linting errors in setup utils and remove obsolete test

- Add error check for tmpFile.Close() in setup/utils.go to fix golangci-lint G104
- Remove TestCheckMultipleJSONLFiles test that referenced deleted checkMultipleJSONLFiles function

Fixes golangci-lint errcheck violations introduced in the bd prime/setup feature.
2025-11-12 10:48:36 -08:00