From 174ff3270d8e2aa874a76409083820f27c188050 Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Wed, 15 Oct 2025 01:56:51 -0700 Subject: [PATCH] fix: remove legacy beads.db and file import timeout bug (bd-199) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Deleted stale beads.db (last modified Oct 14, 20KB) - bd.db is the active database (2MB, 208 issues) - Filed bd-199 to investigate import timeout with 208 issues - Import should handle 100k+ issues, currently times out at 208 šŸ¤– Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .beads/issues.jsonl | 1 + 1 file changed, 1 insertion(+) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 9a5a02d0..831d8c25 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -108,6 +108,7 @@ {"id":"bd-196","title":"Test2 auto-export","description":"","status":"closed","priority":4,"issue_type":"task","created_at":"2025-10-15T01:08:12.788761-07:00","updated_at":"2025-10-15T01:08:12.788761-07:00"} {"id":"bd-197","title":"Add version mismatch warning for outdated binaries","description":"When bd detects that the database was created/modified by a newer version, warn the user that they may be running an outdated binary.\n\n**Problem:** Users may run old bd binaries without realizing it, leading to confusing behavior (e.g., missing features like auto-export).\n\n**Solution:** Store schema version or bd version in metadata table, check on startup, warn if mismatch.\n\n**Implementation:**\n1. Add `schema_version` or `bd_version` to metadata table during init\n2. Check version in PersistentPreRun (cmd/bd/main.go)\n3. Warn if binary version \u003c DB version\n4. Suggest: 'Your bd binary (v0.9.3) is older than this database (v0.9.5). Rebuild: go build -o bd ./cmd/bd'\n\n**Edge cases:**\n- Dev builds (show commit hash?)\n- Backwards compatibility (older DBs should work with newer binaries)\n- Don't warn on every command (maybe once per session?)\n\n**Related:** Fixes confusion from bd-182 (auto-export not working with old binary)","notes":"**Implementation Complete:**\n\nāœ… Added version metadata storage during `bd init` (init.go:59-63)\nāœ… Added `checkVersionMismatch()` function (main.go:301-345)\nāœ… Integrated version check into PersistentPreRun (main.go:98-99)\nāœ… Tested both scenarios:\n - Outdated binary: Clear warning with rebuild instructions\n - Newer binary: Info message that DB will be auto-upgraded\nāœ… No warnings on subsequent runs (version updated automatically)\n\n**How it works:**\n1. On `bd init`: Stores current version in metadata table\n2. On every command: Checks stored version vs binary version\n3. If mismatch:\n - Binary \u003c DB version ā†’ Warn: outdated binary\n - Binary \u003e DB version ā†’ Info: auto-upgrading DB\n4. Always updates stored version to current (future-proof)\n\n**Files modified:**\n- cmd/bd/init.go: Store version on init\n- cmd/bd/main.go: checkVersionMismatch() + integration\n\n**Testing:**\n```bash\n# Simulate old binary\nsqlite3 .beads/default.db \"UPDATE metadata SET value='0.9.99' WHERE key='bd_version';\"\nbd ready # Shows warning\n\n# Normal use (versions match)\nbd ready # No warning\n```\n\n**Edge cases handled:**\n- Empty version (old DB): Silently upgrade\n- Metadata errors: Skip check (defensive)\n- Dev builds: String comparison (works for 0.9.5 vs 0.9.6)\n\nFixes bd-182 confusion (users won't run outdated binaries unknowingly).","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-15T01:08:12.788913-07:00","updated_at":"2025-10-15T01:08:12.788913-07:00","dependencies":[{"issue_id":"bd-197","depends_on_id":"bd-182","type":"discovered-from","created_at":"2025-10-15T01:08:12.811298-07:00","created_by":"auto-import"}]} {"id":"bd-198","title":"Add label management commands to CLI","description":"Currently labels can only be managed programmatically via the storage API. Add CLI commands to add, remove, and filter by labels.","acceptance_criteria":"Can add/remove/list labels via CLI, can filter issues by label, labels persist to JSONL","status":"open","priority":2,"issue_type":"feature","created_at":"2025-10-15T01:50:18.575625-07:00","updated_at":"2025-10-15T01:50:18.575625-07:00"} +{"id":"bd-199","title":"Investigate and fix import timeout with 208 issues","description":"bd import times out after 2 minutes when importing 208 issues from JSONL. This is unacceptable for a tool designed to scale to 100k+ issues.","design":"\n## Reproduction\n```bash\ncd ~/src/beads\nbd import .beads/issues.jsonl # Hangs/times out after 2 minutes\n```\n\nCurrent database state:\n- 208 issues in bd.db (2MB)\n- 208 lines in issues.jsonl (100KB)\n- WAL mode enabled\n\n## Symptoms\n- Import starts but never completes\n- No error message, just hangs\n- Timeout after 2 minutes (artificially imposed in testing)\n- Happens even when database already contains the issues (idempotent import)\n\n## Likely Causes\n1. **Transaction size** - All 208 issues in one transaction?\n2. **Lock contention** - WAL checkpoint blocking?\n3. **N+1 queries** - Dependency checking for each issue?\n4. **Missing indexes** - Slow lookups during collision detection?\n5. **Auto-flush interaction** - Background flush goroutine conflicting?\n\n## Performance Target\nShould handle:\n- 1000 issues in \u003c 5 seconds\n- 10,000 issues in \u003c 30 seconds \n- 100,000 issues in \u003c 5 minutes\n\n## Investigation Steps\n1. Add timing instrumentation to import phases\n2. Profile the import operation\n3. Check for lock contention in WAL mode\n4. Review collision detection performance\n5. Test with PRAGMA synchronous = NORMAL\n6. Consider batching imports (e.g., 100 issues per transaction)\n","acceptance_criteria":"\n- Can import 208 issues in \u003c 5 seconds\n- Can import 1000 issues in \u003c 30 seconds\n- Add performance logging to identify bottlenecks\n- Add --batch-size flag for tuning\n- Document performance characteristics\n- Integration test with large import\n","status":"open","priority":0,"issue_type":"bug","created_at":"2025-10-15T01:56:41.546786-07:00","updated_at":"2025-10-15T01:56:41.546786-07:00"} {"id":"bd-2","title":"Verify auto-export works","description":"","status":"closed","priority":0,"issue_type":"task","created_at":"2025-10-12T00:43:03.457453-07:00","updated_at":"2025-10-15T01:08:12.789064-07:00","closed_at":"2025-10-12T14:15:04.00695-07:00"} {"id":"bd-20","title":"Add --strict flag for dependency import failures","description":"Currently dependency import errors are warnings (logged to stderr, execution continues). Missing targets or cycles may indicate JSONL corruption. Add --strict flag to fail on any dependency errors for data integrity validation. Location: cmd/bd/import.go:159-164","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-10-12T15:13:18.954834-07:00","updated_at":"2025-10-15T01:08:12.789192-07:00"} {"id":"bd-21","title":"Simplify getNextID SQL query parameters","description":"Query passes prefix four times to same SQL query. Works but fragile if query changes. Consider simplifying SQL to require fewer parameters. Location: internal/storage/sqlite/sqlite.go:73-78","status":"closed","priority":3,"issue_type":"task","created_at":"2025-10-12T15:13:20.114733-07:00","updated_at":"2025-10-15T01:08:12.789328-07:00"}