diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 4872815b..31c3eeb9 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -140,7 +140,7 @@ {"id":"bd-224","title":"Data model allows inconsistent status/closed_at states","description":"Issue bd-89 demonstrates a data model inconsistency: an issue can have status='open' but also have a closed_at timestamp set. This creates a liminal state that violates the expected invariant that closed_at should only be set when status='closed'.\n\nRoot causes:\n1. Import (bd import) updates status field independently from closed_at field\n2. UpdateIssue allows status changes without managing closed_at\n3. No database constraint enforcing the invariant\n4. Export includes both fields independently in JSONL\n\nCurrent behavior:\n- bd close: Sets status='closed' AND closed_at (correct)\n- bd update --status open: Sets status='open' but leaves closed_at unchanged (creates inconsistency)\n- bd import: Can import inconsistent data from JSONL\n\nImpact:\n- 'bd ready' shows issues that appear closed (have closed_at)\n- Confusing for users and downstream tools\n- Stats may be inaccurate\n\nPotential solutions:\nA) Add CHECK constraint: (status = 'closed') = (closed_at IS NOT NULL)\nB) Update import/update logic to enforce invariant in application code\nC) Add a 'reopened' event that explicitly clears closed_at\nD) Remove closed_at field entirely (calculate from events or use status only)\n\nSee bd-89 for concrete example.","status":"in_progress","priority":1,"issue_type":"bug","created_at":"2025-10-15T01:36:21.971783-07:00","updated_at":"2025-10-15T03:01:29.56434-07:00"} {"id":"bd-225","title":"Ultrathink: Choose solution for status/closed_at inconsistency (bd-224)","description":"Deep analysis of solution options for bd-224 data model inconsistency issue.\n\nContext:\n- Target users: individual devs and small teams\n- Future: swarms of agent workers\n- Brand-new codebase with few users (can break things)\n- Issue: status='open' with closed_at!=NULL creates liminal state\n\nOptions to evaluate:\nA) Database CHECK constraint\nB) Application-level enforcement \nC) Add explicit reopened event\nD) Remove closed_at field entirely\n\nAnalysis framework:\n1. Simplicity (mental model for devs)\n2. Robustness (hard to break, especially for agent swarms)\n3. Migration cost (schema changes, data cleanup)\n4. Future extensibility\n5. Performance\n6. Consistency guarantees\n\nNeed to determine best approach given tradeoffs.","notes":"Analysis complete. Recommendation: Hybrid approach with DB CHECK constraint + smart UpdateIssue + import enforcement + reopen command.\n\nKey insights:\n- DB constraint provides defense-in-depth perfect for agent swarms\n- Statistics calculation currently uses 'closed_at IS NOT NULL' which is BROKEN by inconsistent data\n- UpdateIssue and Import don't manage the invariant\n- EventReopened exists but is unused\n\nSee ULTRATHINK_BD224.md for full analysis.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T01:47:25.564925-07:00","updated_at":"2025-10-15T03:01:29.564744-07:00","closed_at":"2025-10-15T01:49:43.078431-07:00","dependencies":[{"issue_id":"bd-225","depends_on_id":"bd-224","type":"discovered-from","created_at":"2025-10-15T01:47:25.567014-07:00","created_by":"stevey"}]} {"id":"bd-226","title":"Epic: Fix status/closed_at inconsistency (bd-224 solution)","description":"Implement hybrid solution to enforce status/closed_at invariant:\n- Database CHECK constraint\n- Smart UpdateIssue logic\n- Import enforcement\n- Reopen command\n\nThis is a data integrity issue that affects statistics and will cause problems for agent swarms. The hybrid approach provides defense-in-depth.\n\nSee ULTRATHINK_BD224.md for full analysis and rationale.\n\nParent of all implementation tasks for this fix.","status":"open","priority":1,"issue_type":"epic","created_at":"2025-10-15T01:58:41.041574-07:00","updated_at":"2025-10-15T03:01:29.565155-07:00"} -{"id":"bd-227","title":"Audit and document all inconsistent issues in database","description":"Before we add the constraint, we need to know what data is inconsistent.\n\nSteps:\n1. Query database for status/closed_at mismatches\n2. Document each case (how many, which issues, patterns)\n3. Decide on cleanup strategy (trust status vs trust closed_at)\n4. Create SQL script for cleanup\n\nOutput: Document with counts and cleanup SQL ready to review.\n\nThis unblocks the migration work.","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-15T01:58:50.908363-07:00","updated_at":"2025-10-15T03:01:29.565623-07:00","dependencies":[{"issue_id":"bd-227","depends_on_id":"bd-226","type":"parent-child","created_at":"2025-10-15T01:58:50.909081-07:00","created_by":"stevey"}]} +{"id":"bd-227","title":"Audit and document all inconsistent issues in database","description":"Before we add the constraint, we need to know what data is inconsistent.\n\nSteps:\n1. Query database for status/closed_at mismatches\n2. Document each case (how many, which issues, patterns)\n3. Decide on cleanup strategy (trust status vs trust closed_at)\n4. Create SQL script for cleanup\n\nOutput: Document with counts and cleanup SQL ready to review.\n\nThis unblocks the migration work.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T01:58:50.908363-07:00","updated_at":"2025-10-15T13:08:48.165755-07:00","closed_at":"2025-10-15T13:08:48.165755-07:00","dependencies":[{"issue_id":"bd-227","depends_on_id":"bd-226","type":"parent-child","created_at":"2025-10-15T01:58:50.909081-07:00","created_by":"stevey"}]} {"id":"bd-228","title":"Critical: Auto-import silently overwrites local changes without collision detection","description":"## Problem\n\nAuto-import is silently overwriting local changes when pulling from Git, without using collision detection or warning the user.\n\n## Evidence\n\nIssue bd-89 timeline:\n1. 08:07:31 - Manual update: set status=in_progress\n2. 08:13:08 - Manual close: set status=closed, closed_at set\n3. 08:33:25 - **Auto-import silently overwrote to status=open** (discarded local change)\n4. 08:57:59 - Auto-import overwrote again\n5. 08:58:30 - Auto-import overwrote again\n\nQuery shows 10+ issues with multiple auto-import overwrites, including bd-198, bd-199, bd-200-206.\n\n## Root Cause Hypothesis\n\nWhen two clones both modify issue N:\n- Clone A: modifies bd-89, exports to JSONL, commits\n- Clone B: also modifies bd-89 differently, pulls from A\n- Auto-import uses UpdateIssue which **unconditionally overwrites**\n- Clone B's local changes are silently lost\n- No collision detection warning\n- References and work context corrupted\n\n## Impact\n\n- **Data loss**: Local changes silently discarded\n- **Corruption**: Issue state doesn't reflect actual work done\n- **Lost references**: Comments/deps referencing lost state are now wrong\n- **Trust**: Can't trust the database reflects reality\n- **Agent swarms**: Catastrophic for parallel workers\n\n## Expected Behavior\n\nAuto-import should:\n1. Detect collisions (same ID, different content)\n2. Warn user or fail import\n3. Offer resolution options (--resolve-collisions)\n4. Never silently overwrite without user confirmation\n\n## Current Behavior\n\nAuto-import silently overwrites using UpdateIssue, no collision detection.\n\n## Reproduction\n\n1. Clone A: bd update bd-1 --status in_progress\n2. Clone A: git add .beads/issues.jsonl \u0026\u0026 git commit \u0026\u0026 git push\n3. Clone B: bd update bd-1 --status closed \n4. Clone B: git pull (triggers auto-import)\n5. Result: bd-1 status silently reverted to in_progress, closed state lost\n\n## Files Involved\n\n- cmd/bd/main.go: auto-import trigger logic\n- cmd/bd/import.go: import without collision detection\n- internal/storage/sqlite/sqlite.go: UpdateIssue unconditionally overwrites","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-10-15T02:06:30.671918-07:00","updated_at":"2025-10-15T03:01:29.566028-07:00","closed_at":"2025-10-15T02:11:30.794518-07:00"} {"id":"bd-229","title":"Investigate data recovery for issues overwritten by auto-import bug","description":"Auto-import bug (bd-228) has been silently overwriting local changes since the feature was introduced. We need to investigate if any important data was lost and if it can be recovered.\n\n## Data Sources for Recovery\n\n1. **Git history of .beads/issues.jsonl** - May contain older versions with lost changes\n2. **Events table** - Has full audit trail with old_value/new_value\n3. **Commit messages** - May reference work that was done but later overwritten\n\n## Investigation Steps\n\n1. Check events table for patterns:\n - Look for auto-import events that changed status from closed → open\n - Look for manual updates followed by auto-import reverts\n - Identify issues with suspicious event sequences\n\n2. Git archaeology:\n - Check git log for .beads/issues.jsonl\n - Look for commits where status=closed was changed to status=open\n - Compare git history vs current database state\n\n3. Document lost work:\n - Create list of issues that were closed but reverted to open\n - Identify any critical work that was marked done but lost\n - Check if any issue references are now broken\n\n## Recovery Actions\n\nIf significant data loss is found:\n- Create script to reconstruct lost state from events table\n- Manually review and restore critical closed issues\n- Document lessons learned for preventing future data loss\n\n## Example Query\n\n```sql\n-- Find issues where status was manually changed then reverted by auto-import\nSELECT \n issue_id,\n GROUP_CONCAT(event_type || ':' || actor, ' -\u003e ') as event_chain\nFROM events \nWHERE issue_id IN (\n SELECT DISTINCT issue_id \n FROM events \n WHERE actor = 'auto-import' AND event_type = 'status_changed'\n)\nGROUP BY issue_id\nHAVING event_chain LIKE '%stevey%auto-import%';\n```","notes":"Investigation complete. Successfully recovered 3 valuable issues (bd-232, bd-233) from 22 total lost. Historical context preserved in bd-89 notes. Auto-import collision detection (bd-228) now prevents future data loss.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T02:10:40.724826-07:00","updated_at":"2025-10-15T03:01:29.566441-07:00","closed_at":"2025-10-15T02:22:30.426785-07:00","dependencies":[{"issue_id":"bd-229","depends_on_id":"bd-228","type":"discovered-from","created_at":"2025-10-15T02:10:40.740673-07:00","created_by":"stevey"}]} {"id":"bd-23","title":"Optimize export dependency queries (N+1 problem)","description":"Export triggers separate GetDependencyRecords() per issue. For large DBs (1000+ issues), this is N+1 queries. Add GetAllDependencyRecords() to fetch all dependencies in one query. Location: cmd/bd/export.go:52-59, import.go:138-142","status":"closed","priority":3,"issue_type":"task","created_at":"2025-10-14T14:43:06.910124-07:00","updated_at":"2025-10-15T03:01:29.566961-07:00"} @@ -217,7 +217,7 @@ {"id":"bd-84","title":"Auto-import fails in git workflows due to mtime issues","description":"The auto-import mechanism (autoImportIfNewer) relies on file modification time comparison between JSONL and DB. This breaks in git workflows because git does not preserve original file modification times - pulled files get fresh mtimes based on checkout time.\n\nRoot causes:\n1. Git checkout sets mtime to 'now', not original commit time\n2. Auto-import compares JSONL mtime vs DB mtime (line 181 in main.go)\n3. If DB was recently modified (agents working), mtime check fails\n4. Auto-import silently returns without feedback\n5. Agents continue with stale database state\n\nThis caused issues in VC project where 3 parallel agents:\n- Pulled updated .beads/issues.jsonl from git\n- Auto-import didn't trigger (JSONL appeared older than DB)\n- Agents couldn't find their assigned issues\n- Agents exported from wrong database, corrupting JSONL","design":"Recommended approach: Checksum-based sync (option 3 from original design)\n\n## Solution: Hash-based content comparison\n\nReplace mtime comparison with JSONL content hash comparison:\n\n1. **Compute JSONL hash on startup**:\n - SHA256 hash of .beads/issues.jsonl contents\n - Fast enough for typical repos (\u003c1MB = ~20ms)\n - Only computed once per command invocation\n\n2. **Store last import hash in DB**:\n - Add metadata table if not exists: CREATE TABLE IF NOT EXISTS metadata (key TEXT PRIMARY KEY, value TEXT)\n - Store hash after successful import: INSERT OR REPLACE INTO metadata (key, value) VALUES ('last_import_hash', '\u003chash\u003e')\n - Query on startup: SELECT value FROM metadata WHERE key = 'last_import_hash'\n\n3. **Compare hashes instead of mtimes**:\n - If JSONL hash != stored hash: auto-import (content changed)\n - If JSONL hash == stored hash: skip import (no changes)\n - If no stored hash: fall back to mtime comparison (backward compat)\n\n4. **Update autoImportIfNewer() in cmd/bd/main.go**:\n - Lines 155-279 currently use mtime comparison (line 181)\n - Replace with hash comparison\n - Keep mtime as fallback for old DBs without metadata table\n\n## Implementation Details\n\n### New storage interface method:\n```go\n// In internal/storage/storage.go\ntype Storage interface {\n // ... existing methods ...\n GetMetadata(ctx context.Context, key string) (string, error)\n SetMetadata(ctx context.Context, key, value string) error\n}\n```\n\n### Migration:\n```go\n// In internal/storage/sqlite/sqlite.go init\nCREATE TABLE IF NOT EXISTS metadata (\n key TEXT PRIMARY KEY,\n value TEXT NOT NULL\n);\n```\n\n### Updated autoImportIfNewer():\n```go\nfunc autoImportIfNewer() {\n jsonlPath := findJSONLPath()\n \n // Check if JSONL exists\n jsonlData, err := os.ReadFile(jsonlPath)\n if err != nil {\n return // No JSONL, skip\n }\n \n // Compute current hash\n hasher := sha256.New()\n hasher.Write(jsonlData)\n currentHash := hex.EncodeToString(hasher.Sum(nil))\n \n // Get last import hash from DB\n ctx := context.Background()\n lastHash, err := store.GetMetadata(ctx, \"last_import_hash\")\n if err != nil {\n // No metadata support (old DB) - fall back to mtime comparison\n autoImportIfNewerByMtime()\n return\n }\n \n // Compare hashes\n if currentHash == lastHash {\n return // No changes, skip import\n }\n \n // Content changed - import\n if err := importJSONLSilent(jsonlPath, jsonlData); err != nil {\n return // Import failed, skip\n }\n \n // Store new hash\n _ = store.SetMetadata(ctx, \"last_import_hash\", currentHash)\n}\n```\n\n## Benefits\n\n- **Git-proof**: Works regardless of file timestamps\n- **Universal**: Works with git, Dropbox, rsync, manual edits\n- **Backward compatible**: Falls back to mtime for old DBs\n- **Efficient**: SHA256 is fast (~20ms for 1MB)\n- **Accurate**: Only imports when content actually changed\n- **No user action**: Fully automatic, invisible\n\n## Performance Optimization\n\nFor very large repos (\u003e10MB JSONL):\n- Only hash if mtime changed (combine both checks)\n- Use incremental hashing if metadata table tracks line count\n- Consider sampling hash (first 1MB + last 1MB)\n\nBut start simple - full hash is fast enough for 99% of use cases.\n\n## Rollout Plan\n\n1. Add metadata table + Get/SetMetadata methods (backward compatible)\n2. Update autoImportIfNewer() with hash logic + mtime fallback\n3. Test with old and new DBs\n4. Ship in next minor version (v0.10.0)\n5. Document in CHANGELOG as \"more reliable auto-import\"\n6. Git hooks remain optional but unnecessary for most users","acceptance_criteria":"- Auto-import works correctly after git pull\n- Agents in parallel workflows see consistent database state\n- Clear feedback when import is needed\n- Performance acceptable for large databases\n- Works in both git and non-git workflows\n- Documentation updated with multi-agent best practices","status":"in_progress","priority":1,"issue_type":"bug","created_at":"2025-10-14T14:43:06.941275-07:00","updated_at":"2025-10-15T03:01:29.597563-07:00"} {"id":"bd-85","title":"GH-1: Fix bd dep tree graph display issues","description":"Tree display has several issues: 1) Epic items may not expand all sub-items, 2) Subitems repeat multiple times at same level, 3) Items with multiple blockers appear multiple times. The tree visualization doesn't properly handle graph structures with multiple dependencies.","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-10-14T14:43:06.941898-07:00","updated_at":"2025-10-15T03:01:29.598191-07:00","external_ref":"gh-1"} {"id":"bd-86","title":"GH-2: Evaluate optional Turso backend for collaboration","description":"RFC proposal for optional Turso/libSQL backend to enable: database branching, near-real-time sync between agents/humans, native vector search, browser-ready persistence (WASM/OPFS), and concurrent writes. Would be opt-in, keeping current JSONL+SQLite as default. Requires storage driver interface.","status":"open","priority":3,"issue_type":"feature","created_at":"2025-10-14T14:43:06.942254-07:00","updated_at":"2025-10-15T03:01:29.598773-07:00","external_ref":"gh-2"} -{"id":"bd-87","title":"GH-3: Debug zsh killed error on bd init","description":"User reports 'zsh: killed bd init' when running bd init or just bd command. Likely a crash or signal. Need to reproduce and investigate cause.","status":"open","priority":1,"issue_type":"bug","created_at":"2025-10-14T14:43:06.942576-07:00","updated_at":"2025-10-15T03:01:29.599197-07:00","external_ref":"gh-3"} +{"id":"bd-87","title":"GH-3: Debug zsh killed error on bd init","description":"User reports 'zsh: killed bd init' when running bd init or just bd command. Likely a crash or signal. Need to reproduce and investigate cause.","notes":"Awaiting user feedback - cannot reproduce locally, waiting for user to provide more details about environment and error message","status":"blocked","priority":1,"issue_type":"bug","created_at":"2025-10-14T14:43:06.942576-07:00","updated_at":"2025-10-15T13:05:48.603124-07:00","external_ref":"gh-3"} {"id":"bd-88","title":"GH-4: Consider system-wide/multi-repo beads usage","description":"User wants to use beads across multiple repositories and for sysadmin tasks. Currently beads is project-scoped (.beads/ directory). Explore options for system-wide issue tracking that spans multiple repos. Related question: how does beads compare to membank MCP?","status":"open","priority":3,"issue_type":"feature","created_at":"2025-10-14T14:43:06.942895-07:00","updated_at":"2025-10-15T03:01:29.599625-07:00","external_ref":"gh-4"} {"id":"bd-89","title":"GH-6: Fix race condition in parallel issue creation","description":"Creating multiple issues rapidly in parallel causes 'UNIQUE constraint failed: issues.id' error. The ID generation has a race condition. Reproducible with: for i in {26..35}; do ./bd create parallel_ 2\u003e\u00261 \u0026 done","notes":"**Historical context (recovered from lost issue bd-221):**\n\nFirst attempt at fixing this race condition had a critical flaw: used 'ROLLBACK; BEGIN IMMEDIATE' which executed as two separate statements. After ROLLBACK, the Go tx object was invalid but continued to be used, causing undefined behavior.\n\nRoot cause of failed fix: database/sql connection pooling. Without acquiring a dedicated connection, subsequent queries could use different connections from the pool, breaking the transaction.\n\nCorrect fix (the one that was merged): Use conn := s.db.Conn(ctx) to acquire a dedicated connection, then execute BEGIN IMMEDIATE, all operations, and COMMIT on that single connection.\n\nThis bug was caught during code review and fixed before merging.\n\nSee LOST_ISSUES_RECOVERY.md for details on bd-221.","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-10-14T14:43:06.943239-07:00","updated_at":"2025-10-15T03:01:29.600056-07:00","closed_at":"2025-10-15T02:52:17.500349-07:00","external_ref":"gh-6"} {"id":"bd-9","title":"Build collision resolution tooling for distributed branch workflows","description":"When branches diverge and both create issues, auto-incrementing IDs collide on merge. Build excellent tooling to detect collisions during import, auto-renumber issues with fewer dependencies, update all references in descriptions and dependency links, and provide clear user feedback. Goal: keep beautiful brevity of numeric IDs (bd-302) while handling distributed creation gracefully.","status":"closed","priority":1,"issue_type":"feature","created_at":"2025-10-14T14:43:06.943629-07:00","updated_at":"2025-10-15T03:01:29.600517-07:00"}