Commit Graph

16 Commits

Author SHA1 Message Date
Steve Yegge
40b07045c7 refactor: deduplicate FindJSONLInDir function (bd-8a5)
Extract shared JSONL file discovery logic to internal/utils/path.go.
Both autoimport and beads packages now use this shared implementation.

Changes:
- Add utils.FindJSONLInDir with common logic
- Update autoimport.go to use utils.FindJSONLInDir
- Update beads.go to delegate to utils.FindJSONLInDir
- Update server_export_import_auto.go to use utils.FindJSONLInDir
- Move FindJSONLInDir test to utils/path_test.go
- Fix pre-existing duplicate countIssuesInJSONLFile in init.go

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 23:07:53 -08:00
Steve Yegge
887c958567 fix(autoimport): prevent export to wrong JSONL file (bd-tqo)
Add FindJSONLInDir helper that correctly prefers issues.jsonl over other
.jsonl files. Previously, glob patterns could return deletions.jsonl or
merge artifacts (beads.base.jsonl, etc.) first alphabetically, causing
issue data to be written to the wrong file.

This fixes the root cause of deletions.jsonl corruption where full issue
objects were written instead of deletion records, leading to all issues
being purged during sync.

Changes:
- Add FindJSONLInDir() in internal/autoimport with proper file selection
- Update AutoImportIfNewer() to use FindJSONLInDir
- Update CheckStaleness() to use FindJSONLInDir
- Update triggerExport() in RPC server to use FindJSONLInDir
- Add comprehensive tests for FindJSONLInDir

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 23:25:32 -08:00
Steve Yegge
e3e0a04496 Add configurable export error handling policies (bd-exug)
Implements flexible error handling for export operations with four policies:
- strict: Fail-fast on any error (default for user exports)
- best-effort: Skip errors with warnings (default for auto-exports)
- partial: Retry then skip with manifest tracking
- required-core: Fail on core data, skip enrichments

Key features:
- Per-project configuration via `bd config set export.error_policy`
- Separate policy for auto-exports: `auto_export.error_policy`
- Retry with exponential backoff (configurable attempts/delay)
- Optional export manifests documenting completeness
- Per-issue encoding error handling

This allows users to choose the right trade-off between data integrity
and system availability for their specific project needs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 20:21:51 -08:00
Steve Yegge
9c6b37500c Fix N+1 query pattern in export operations (bd-rcmg)
**Problem**: Export operations called GetLabels() and GetIssueComments()
in a loop for each issue, creating N+1 query pattern. For 100 issues
this created 201 queries instead of 3-5.

**Solution**:
- Added GetCommentsForIssues() batch method to storage interface
- Implemented batch method in SQLite and memory storage backends
- Updated handleExport() and triggerExport() to use batch queries
- Added comprehensive tests for batch operations

**Impact**: Query count reduced from ~201 to ~3-5 for 100 issues.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 19:53:44 -08:00
Steve Yegge
012bed1068 Fix GH#367: bd import defaulting to stdin is confusing
Changes:
1. Add TTY detection to bd import - prevents silent hang when run
   interactively without arguments. Shows helpful usage message instead.
2. Fix misleading error messages - change "Run 'bd import'" to
   "Run 'bd sync --import-only'" or explicit file path throughout.

Technical details:
- Added golang.org/x/term dependency for IsTerminal()
- When stdin is a TTY and no -i flag: show usage and exit
- When stdin is piped: works as before (supports script pipelines)
- Preserved all legitimate stdin uses:
  * python gh2jsonl.py --repo owner/repo | bd import
  * python md2jsonl.py feature.md | bd import
  * git show HEAD:.beads/beads.jsonl | bd import -i /dev/stdin

Updated error messages in:
- cmd/bd/staleness.go - main "out of sync" error
- cmd/bd/sync.go - merge completion suggestions
- internal/rpc/server_export_import_auto.go - daemon warnings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 11:15:34 -08:00
Steve Yegge
9088988edd Improve staleness check error handling and optimization (bd-n4td, bd-o4qy, bd-c4rq)
This commit implements three related improvements to database staleness checking:

**bd-n4td (P2): Add warning when staleness check errors**
- Added stderr warnings when CheckStaleness fails in ensureDatabaseFresh
- Users now see "Warning: could not check database staleness: <error>"
- Provides visibility into staleness check failures while allowing operations to proceed

**bd-o4qy (P2): Improve CheckStaleness error handling**
- Updated CheckStaleness to distinguish between expected and abnormal conditions:
  * Returns (false, nil) for expected "no data yet" scenarios (missing metadata, missing JSONL)
  * Returns (false, err) for abnormal errors (glob failures, permission errors)
- Updated RPC server (2 locations) to log staleness errors but allow requests to proceed
- Prevents blocking operations due to transient staleness check issues
- Added comprehensive function documentation

**bd-c4rq (P3): Refactor staleness check to avoid function call overhead**
- Moved daemon check from inside ensureDatabaseFresh to all 8 call sites
- Avoids unnecessary function call when running in daemon mode
- Updated: list.go, info.go, status.go, show.go, stale.go, duplicates.go, ready.go, validate.go
- Extracted staleness functions to new staleness.go for better organization

**Code review fixes:**
- Removed dead code in CheckStaleness (unreachable jsonlPath == "" check)
- Removed unused ensureDatabaseFreshQuiet function

**Files changed:**
- New: cmd/bd/staleness.go (extracted staleness checking functions)
- Modified: 8 command files (added daemon check before staleness calls)
- Modified: internal/autoimport/autoimport.go (improved error handling)
- Modified: internal/rpc/server_export_import_auto.go (handle errors gracefully)
2025-11-20 20:45:39 -05:00
Steve Yegge
95cbcf4fbc Centralize BD_DEBUG logging into internal/debug package
- Created internal/debug package with Enabled(), Logf(), Printf()
- Added comprehensive unit tests for debug package
- Replaced 50+ scattered os.Getenv("BD_DEBUG") checks across 9 files
- Centralized debug logic for easier maintenance and testing
- All tests passing, behavior unchanged

Closes bd-fb95094c.5
2025-11-06 20:14:34 -08:00
Steve Yegge
481649a605 Fix widespread double JSON encoding bug in daemon RPC calls (bd-1048, bd-4ec8)
Multiple CLI commands had a systematic bug where ResolveID responses were
incorrectly converted using string(resp.Data) instead of json.Unmarshal.
Since resp.Data is json.RawMessage (already JSON-encoded), this preserved
the JSON quotes, causing IDs to become "bd-1048" instead of bd-1048.
When re-marshaled for subsequent RPC calls, these became double-quoted
("\"bd-1048\""), causing database lookups to fail.

Bugs fixed:
1. Nil pointer dereference in handleShow - added nil check after GetIssue
2. Double JSON encoding in 12 locations across 4 commands:
   - bd show (3 instances in show.go)
   - bd dep add/remove/tree (5 instances in dep.go)
   - bd label add/remove/list (3 instances in label.go)
   - bd reopen (1 instance in reopen.go)

All instances replaced string(resp.Data) with proper json.Unmarshal.
Removed debug logging added during investigation.

Tested: All affected commands now work correctly with daemon mode.
2025-11-02 22:34:24 -08:00
Steve Yegge
63ff9b93bc Update bd-1048 with investigation findings 2025-11-02 22:12:21 -08:00
Steve Yegge
86c645603e Fix all gosec, misspell, and unparam linter errors
- Add #nosec directives with explanations for all gosec warnings in worktree operations
- Tighten directory permissions from 0755 to 0750 for better security
- Fix misspellings: archaeological -> archeological, cancelled -> canceled
- Remove unused jsonlPath parameter from syncBranchCommitAndPush
- Change branchExists to return bool instead of (bool, error) - error was never used

All changes maintain backward compatibility and improve code quality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 20:06:05 -08:00
Steve Yegge
3b6856904f Fix critical double-release race in importInProgress flag
CRITICAL BUG: The previous fix had a race condition where the
importInProgress flag could be released twice, allowing two goroutines
to think they both hold the lock.

Bug scenario:
1. Goroutine A: acquires lock (CAS true)
2. Goroutine A: manually releases at line 208 for git dirty skip
3. Goroutine B: CAS succeeds, acquires lock
4. Goroutine A: defer runs, releases flag AGAIN (clears B lock)
5. Goroutine C: CAS succeeds - now TWO goroutines have lock

Root cause: Using both manual Store(false) AND defer Store(false)
created a window where the flag could be cleared twice.

Fix: Use a shouldDeferRelease flag to disable the deferred release
when we manually release early. This ensures exactly one release
per acquisition.

Testing: All auto-import tests still passing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 18:16:25 -08:00
Steve Yegge
e8e9e729e5 Fix bd-8931: Prevent daemon corruption from git conflicts
CRITICAL FIX: The daemon could enter a corrupt state when auto-import
was triggered while uncommitted changes existed in .beads/ files. This
caused mysterious EOF errors requiring daemon restarts.

Root Causes Fixed:
1. No git dirty check before auto-import
2. Missing timeout protection (could hang indefinitely)
3. Race condition in importInProgress flag management
4. Improper git status parsing (false positives)
5. Context leak in export goroutine (accessing closed DB)
6. Data loss in triggerExport (missing deps/labels/comments)

Changes:
- Add hasUncommittedBeadsFiles() to check git status before import
  - Properly parses git porcelain format ("XY filename")
  - Ignores untracked files, only checks tracked .jsonl changes
  - 5-second timeout on git command to prevent hangs

- Add 30-second timeout to import operations
  - Prevents daemon from hanging on stuck imports
  - Returns clear error message on timeout

- Fix race condition in importInProgress flag
  - Explicitly release before early returns
  - Prevents other requests seeing incorrect "import in progress"

- Fix context leak in onChanged export goroutine
  - Check daemon shutdown state before export
  - Suppress expected "database closed" errors during shutdown
  - Pass storage/path as parameters to avoid closure issues

- Fix data loss in triggerExport()
  - Populate dependencies, labels, comments (mirrors handleExport)
  - Use atomic file write (temp + rename)
  - Sort issues for consistent output

Impact: Prevents daemon corruption in collaborative workflows where
users frequently pull updates while having local uncommitted changes.

Testing: All auto-import tests passing
- TestDaemonAutoImportAfterGitPull ✓
- TestDaemonAutoImportDataCorruption ✓
- internal/autoimport tests ✓

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-02 18:13:37 -08:00
Steve Yegge
0725c33fcc Remove vestigial --resolve-collisions flag
Hash-based IDs make collision resolution unnecessary. The flag was
already non-functional (handleCollisions returns error on collision
regardless of flag value).

Removed:
- --resolve-collisions flag from bd import
- ResolveCollisions field from ImportOptions and importer.Options
- All references in daemon, auto-import, and tests
- Updated error messages to reflect hash IDs don't collide

All import tests pass.

Amp-Thread-ID: https://ampcode.com/threads/T-47dfa0cc-bb71-4467-ac86-f0966a7c5d58
Co-authored-by: Amp <amp@ampcode.com>
2025-10-31 01:07:42 -07:00
Steve Yegge
bbb1725c32 Replace getStorageForRequest with s.storage (bd-32)
- Replaced all getStorageForRequest(req) calls with direct s.storage access
- Updated 5 handler files: server_issues_epics.go (~8 calls), server_labels_deps_comments.go (~4 calls), server_compact.go (~2 calls), server_export_import_auto.go (~2 calls), server_routing_validation_diagnostics.go (~1 call)
- Only remaining references are in server_cache_storage.go (to be deleted in bd-33) and server_eviction_test.go (to be deleted in bd-34)
- Part of bd-29 epic to remove daemon storage cache

Amp-Thread-ID: https://ampcode.com/threads/T-239a5531-68a5-4c98-b85d-0e3512b2553c
Co-authored-by: Amp <amp@ampcode.com>
2025-10-27 23:14:38 -07:00
Steve Yegge
fb21a319cd Fix bd-28: Auto-import should update issues, not create duplicates
Changed ResolveCollisions from true to false in auto-import logic.
This ensures git pull updates existing issues instead of creating duplicates.
2025-10-27 22:26:44 -07:00
Steve Yegge
0c737025b5 Split internal/rpc/server.go into 8 focused modules (bd-215)
Refactored monolithic 2238-line server.go into 8 files with clear responsibilities:
- server_core.go: Server type, NewServer (115 lines)
- server_lifecycle_conn.go: Start/Stop/connection handling (248 lines)
- server_cache_storage.go: Storage caching and eviction (286 lines)
- server_routing_validation_diagnostics.go: Request routing/validation (384 lines)
- server_issues_epics.go: Issue CRUD operations (506 lines)
- server_labels_deps_comments.go: Labels/deps/comments (199 lines)
- server_compact.go: Compaction operations (287 lines)
- server_export_import_auto.go: Export/import operations (293 lines)

Improvements:
- Replaced RWMutex.TryLock with atomic.Bool for portable single-flight guard
- Added default storage close in Stop() to prevent FD leaks
- All methods remain on *Server receiver (no behavior changes)
- Each file <510 LOC for better maintainability
- All tests pass, daemon verified working

Amp-Thread-ID: https://ampcode.com/threads/T-92d481ad-1bda-4ecd-bcf5-874a1889db30
Co-authored-by: Amp <amp@ampcode.com>
2025-10-27 21:14:34 -07:00