Commit Graph

18 Commits

Author SHA1 Message Date
Steve Yegge
b893be7d0e Fix bd-161: Implement daemon JSONL import (fix NO-OP stub)
Replace the NO-OP importToJSONLWithStore() stub with full implementation:
- Reads and parses JSONL file line by line using bufio.Scanner
- Uses importIssuesCore() with auto-collision resolution enabled
- Integrates with existing import infrastructure
- Fixes PRIMARY root cause of bd-160 multi-clone sync failure

The daemon now properly imports remote changes pulled from git instead
of ignoring them, allowing databases to converge across clones.

Amp-Thread-ID: https://ampcode.com/threads/T-9b92c2dc-e0e2-4d77-b562-136da8c3f64e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-26 20:05:31 -07:00
Steve Yegge
4a340902a8 bd sync: 2025-10-26 19:42:46 2025-10-26 19:42:46 -07:00
Steve Yegge
f3005d3dc8 Export with bd-150/152/153 closed 2025-10-26 19:39:24 -07:00
Steve Yegge
5057cf96ce chore: Remove duplicate bd.jsonl file
The codebase had two identical JSONL files (bd.jsonl and issues.jsonl).
The code defaults to issues.jsonl but FindJSONLPath() would pick bd.jsonl
alphabetically. Removed the duplicate to use only the standard filename.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 14:32:08 -07:00
Steve Yegge
92885bb7a3 feat: Add markdown file support to bd create command
Implement `bd create -f file.md` to parse markdown files and create
multiple issues in one command. This enables drafting features in
markdown and converting them to tracked issues.

Features:
- Parse markdown H2 headers (##) as issue titles
- Support all issue fields via H3 sections (### Priority, ### Type, etc.)
- Handle multiple issues per file
- Comprehensive validation and error handling
- Full test coverage with 5 test cases

Closes bd-91 (GH-9)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 13:21:08 -07:00
Steve Yegge
ccdacf087b fix: Auto-export to JSONL now works correctly [fixes #23]
Fixed bug where PersistentPostRun was clearing isDirty flag before
calling flushToJSONL(), causing the flush to abort immediately.

The fix ensures flushToJSONL() handles the isDirty flag itself,
allowing the JSONL export to complete successfully.

Also added Arch Linux AUR installation instructions to README.

Changes:
- cmd/bd/main.go: Fixed PersistentPostRun flush logic
- README.md: Added Arch Linux (AUR) installation section
- .beads/bd.jsonl: Auto-exported issue bd-169 (init -q flag bug)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 12:35:48 -07:00
Steve Yegge
88b4d710eb feat: Create bd tracking issues for 9 GitHub issues
Created internal tracking issues for all open GitHub Issues:
- bd-85 (gh-1): Fix dep tree graph display issues [P1]
- bd-86 (gh-2): Evaluate Turso backend RFC [P3]
- bd-87 (gh-3): Debug zsh killed error [P1]
- bd-88 (gh-4): System-wide/multi-repo usage [P3]
- bd-89 (gh-6): Fix parallel creation race condition [P0]
- bd-90 (gh-7): AUR package tracking [P4]
- bd-91 (gh-9): Markdown file input support [P2]
- bd-92 (gh-11): Docker/hosted instance support [P2]
- bd-93 (gh-18): Add --deps flag to create [P2]

All issues use the new external_ref field to link to their GitHub
counterparts (gh-N). This establishes proper bidirectional tracking
between our internal beads workflow and public GitHub Issues.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 02:48:00 -07:00
Steve Yegge
6456fe6c02 chore: Update JSONL export after external_ref implementation
Auto-sync export after implementing the external_ref field feature.
All existing issues remain unchanged (field not populated yet).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 02:43:52 -07:00
Steve Yegge
bafb2801c5 Implement incremental JSONL export with dirty issue tracking
Optimize auto-flush by tracking which issues have changed instead of
exporting the entire database on every flush. For large projects with
1000+ issues, this provides significant performance improvements.

Changes:
- Add dirty_issues table to schema with issue_id and marked_at columns
- Implement dirty tracking functions in new dirty.go file:
  * MarkIssueDirty() - Mark single issue as needing export
  * MarkIssuesDirty() - Batch mark multiple issues efficiently
  * GetDirtyIssues() - Query which issues need export
  * ClearDirtyIssues() - Clear tracking after successful export
  * GetDirtyIssueCount() - Monitor dirty issue count
- Update all CRUD operations to mark affected issues as dirty:
  * CreateIssue, UpdateIssue, DeleteIssue
  * AddDependency, RemoveDependency (marks both issues)
  * AddLabel, RemoveLabel, AddEvent
- Modify export to support incremental mode:
  * Add --incremental flag to export only dirty issues
  * Used by auto-flush for performance
  * Full export still available without flag
- Add Storage interface methods for dirty tracking

Performance impact: With incremental export, large databases only write
changed issues instead of regenerating entire JSONL file on every
auto-flush.

Closes bd-39

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 00:17:23 -07:00
Steve Yegge
25644d9717 Cache compiled regexes in ID replacement for 1.9x performance boost
Implements bd-27: Cache compiled regexes in replaceIDReferences for performance

Problem:
replaceIDReferences() was compiling regex patterns on every call. With 100
issues and 10 ID mappings, that resulted in 4,000 regex compilations (100
issues × 4 text fields × 10 ID mappings).

Solution:
- Added buildReplacementCache() to pre-compile all regexes once
- Added replaceIDReferencesWithCache() to reuse compiled regexes
- Updated updateReferences() to build cache once and reuse for all issues
- Kept replaceIDReferences() for backward compatibility (calls cached version)

Performance Results (from benchmarks):
Single text:
- 1.33x faster (26,162 ns → 19,641 ns)
- 68% less memory (25,769 B → 8,241 B)
- 80% fewer allocations (278 → 55)

Real-world (400 texts, 10 mappings):
- 1.89x faster (5.1ms → 2.7ms)
- 90% less memory (7.7 MB → 0.8 MB)
- 86% fewer allocations (104,112 → 14,801)

Tests:
- All existing tests pass
- Added 3 benchmark tests demonstrating improvements

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 23:50:48 -07:00
Steve Yegge
33412871eb Add comprehensive test coverage for auto-flush feature
Implements bd-42: Add test coverage for auto-flush feature

Created cmd/bd/main_test.go with 11 comprehensive test functions:
- TestAutoFlushDirtyMarking: Verifies markDirtyAndScheduleFlush() marks DB as dirty
- TestAutoFlushDisabled: Tests --no-auto-flush flag disables feature
- TestAutoFlushDebounce: Tests rapid operations result in single flush
- TestAutoFlushClearState: Tests clearAutoFlushState() resets state
- TestAutoFlushOnExit: Tests flush happens on program exit
- TestAutoFlushConcurrency: Tests concurrent operations don't cause races
- TestAutoFlushStoreInactive: Tests flush skips when store is inactive
- TestAutoFlushJSONLContent: Tests flushed JSONL has correct content
- TestAutoFlushErrorHandling: Tests error scenarios (permissions, etc.)
- TestAutoImportIfNewer: Tests auto-import when JSONL is newer than DB
- TestAutoImportDisabled: Tests --no-auto-import flag disables auto-import

Coverage results:
- markDirtyAndScheduleFlush: 100%
- clearAutoFlushState: 100%
- flushToJSONL: 67.6%
- autoImportIfNewer: 66.1% (up from 0%)

All tests pass. Auto-flush feature is now thoroughly tested.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 23:38:24 -07:00
Steve Yegge
a8a90e074e Add ID space partitioning and improve auto-flush reliability
Three improvements to beads:

1. ID space partitioning (closes bd-24)
   - Add --id flag to 'bd create' for explicit ID assignment
   - Validates format: prefix-number (e.g., worker1-100)
   - Enables parallel agents to partition ID space and avoid conflicts
   - Storage layer already supported this, just wired up CLI

2. Auto-flush failure tracking (closes bd-38)
   - Track consecutive flush failures with counter and last error
   - Show prominent red warning after 3+ consecutive failures
   - Reset counter on successful flush
   - Users get clear guidance to run manual export if needed

3. Manual export cancels auto-flush timer
   - Add clearAutoFlushState() helper function
   - bd export now cancels pending auto-flush and clears dirty flag
   - Prevents redundant exports when user manually exports
   - Also resets failure counter on successful manual export

Documentation updated in README.md and CLAUDE.md with --id flag examples.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 23:31:51 -07:00
Steve Yegge
252cf9a192 Update issue tracker - close bd-25 as won't-fix
Downgraded bd-25 to P4 and closed as won't-fix for 1.0:
- Transaction support is premature optimization
- SQLite already provides ACID guarantees per-operation
- Collision resolution works reliably without multi-operation transactions
- Would add significant complexity for theoretical benefit

Will revisit if actual issues arise in production use.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 22:59:24 -07:00
Steve Yegge
12a4c384af Fix: Ensure JSONL directory exists in findJSONLPath
Added os.MkdirAll(dbDir, 0755) to ensure the .beads directory exists
before attempting to glob for JSONL files. This fixes a bug where
findJSONLPath() would fail silently if the directory doesn't exist yet,
which can happen during new database initialization.

The fix:
- Creates the directory with 0755 permissions if it doesn't exist
- Handles errors gracefully by returning the default path
- Subsequent write operations will still fail with clear errors if
  directory creation fails

Closes bd-36

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 22:51:04 -07:00
Steve Yegge
584cd1ebfc Implement auto-import to complete automatic git sync workflow
Adds auto-import feature to complement bd-35's auto-export, completing
the automatic sync workflow for git collaboration.

**Implementation:**
- Auto-import checks if JSONL is newer than DB on command startup
- Silently imports JSONL when modification time is newer
- Skips import command itself to avoid recursion
- Can be disabled with --no-auto-import flag

**Documentation updates:**
- Updated README.md git workflow section
- Updated CLAUDE.md workflow and pro tips
- Updated bd quickstart with auto-sync section
- Updated git hooks README to clarify they're now optional

**Testing:**
- Tested auto-import by touching JSONL and running commands
- Tested auto-export with create/close operations
- Complete workflow verified working

Closes bd-33

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 22:48:30 -07:00
Steve Yegge
026940c8ae Track code review findings as issues (bd-36 through bd-42)
Added issues tracking code quality findings from the recent auto-flush implementation:
- bd-36: Handle missing JSONL directory in findJSONLPath
- bd-37: Refactor duplicate flush logic in PersistentPostRun
- bd-38: Add visibility for auto-flush failures
- bd-39: Optimize auto-flush to use incremental updates
- bd-40: Make auto-flush debounce duration configurable
- bd-41: Add godoc comments for auto-flush functions
- bd-42: Add test coverage for auto-flush feature

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 22:37:52 -07:00
Steve Yegge
97d78d264f Fix critical race conditions in auto-flush feature
Fixed three critical issues identified in code review:

1. Race condition with store access: Added storeMutex and storeActive
   flag to prevent background flush goroutine from accessing closed
   store. Background timer now safely checks if store is active before
   attempting flush operations.

2. Missing auto-flush in import: Added markDirtyAndScheduleFlush()
   call after import completes, ensuring imported issues sync to JSONL.

3. Timer cleanup: Explicitly set flushTimer to nil after Stop() to
   prevent resource leaks.

Testing confirmed all fixes working:
- Debounced flush triggers after 5 seconds of inactivity
- Immediate flush on process exit works correctly
- Import operations now trigger auto-flush
- No race conditions detected

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 22:34:12 -07:00
Steve Yegge
15afb5ad17 Implement JSONL export/import and shift to text-first architecture
This is a fundamental architectural shift from binary SQLite to JSONL as
the source of truth for git workflows.

## New Features

- `bd export --format=jsonl` - Export issues to JSON Lines format
- `bd import` - Import issues from JSONL (create new, update existing)
- `--skip-existing` flag for import to only create new issues

## Architecture Change

**Before:** Binary SQLite database committed to git
**After:** JSONL text files as source of truth, SQLite as ephemeral cache

Benefits:
- Git-friendly text format with clean diffs
- AI-resolvable merge conflicts (append-only is 95% conflict-free)
- Human-readable issue tracking in git
- No binary merge conflicts

## Documentation

- Updated README with JSONL-first workflow and git hooks
- Added TEXT_FORMATS.md analyzing JSONL vs CSV vs binary
- Updated GIT_WORKFLOW.md with historical context
- .gitignore now excludes *.db, includes .beads/*.jsonl

## Implementation Details

- Export sorts issues by ID for consistent diffs
- Import handles both creates and updates atomically
- Proper handling of pointer fields (EstimatedMinutes)
- All tests passing

## Breaking Changes

- Database files (*.db) should now be gitignored
- Use export/import workflow for git collaboration
- Git hooks recommended for automation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-12 01:17:50 -07:00