johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

bd sync: 2025-11-01 20:29:22

Browse Source
This commit is contained in:
Steve Yegge
2025-11-01 20:29:23 -07:00
parent 5fabb5fdcc
commit c83560ddd1
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.beads/beads.jsonl
View File

@@ -130,6 +130,7 @@
{"id":"bd-bc2c6191","content_hash":"533e56b8628e24229a4beb52f8683355f6ca699e34a73650bf092003d73c2957","title":"Audit Current Cache Usage","description":"Understand exactly what code depends on the storage cache","acceptance_criteria":"- Document showing all cache dependencies\n- Confirmation that removing cache won't break MCP\n- List of tests that need updating\n\nFiles to examine:\n- internal/rpc/server_cache_storage.go (cache implementation)\n- internal/rpc/client.go (how req.Cwd is set)\n- internal/rpc/server_*.go (all getStorageForRequest calls)\n- integrations/beads-mcp/ (MCP multi-repo logic)\n\nTasks:\n- Document all callers of getStorageForRequest()\n- Verify req.Cwd is only set by RPC client for database discovery\n- Confirm MCP server doesn't rely on multi-repo cache behavior\n- Check if any tests assume multi-repo routing\n- Review environment variables: BEADS_DAEMON_MAX_CACHE_SIZE, BEADS_DAEMON_CACHE_TTL, BEADS_DAEMON_MEMORY_THRESHOLD_MB","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-27T23:02:43.506373-07:00","updated_at":"2025-10-31T20:41:33.916657-07:00"}
{"id":"bd-bdaf24d5","content_hash":"6ccdbf2362d22fbbe854fdc666695a7488353799e1a5c49e6095b34178c9bcb4","title":"Final validation test","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-10-27T18:27:28.310533-07:00","updated_at":"2025-10-31T12:00:43.184995-07:00","closed_at":"2025-10-31T12:00:43.184995-07:00"}
{"id":"bd-c01f","content_hash":"b183cc4d99f74e9314f67d927f1bd1255608632b47ce0352af97af5872275fec","title":"Implement bd stale command to find abandoned/forgotten issues","description":"Add bd stale command to surface issues that haven't been updated recently and may need attention.\n\nUse cases:\n- In-progress issues with no recent activity (may be abandoned)\n- Open issues that have been forgotten\n- Issues that might be outdated or no longer relevant\n\nQuery logic should find non-closed issues where updated_at exceeds a time threshold.\n\nShould support:\n- --days N flag (default 30-90 days)\n- --status filter (e.g., only in_progress)\n- --json output for automation\n\nReferences GitHub issue #184 where user expected this command to exist.","design":"Implementation approach:\n1. Add new command in cmd/bd/stale.go\n2. Query issues with: status != 'closed' AND updated_at \u003c (now - N days)\n3. Support filtering by status (open, in_progress, blocked)\n4. Default threshold: 30 days (configurable via --days)\n5. JSON output for agent consumption\n6. Order by updated_at ASC (oldest first)","status":"closed","priority":2,"issue_type":"epic","created_at":"2025-10-31T22:48:46.85435-07:00","updated_at":"2025-10-31T22:54:33.704492-07:00","closed_at":"2025-10-31T22:54:33.704492-07:00"}
{"id":"bd-c6cf","content_hash":"8760ad512ddc8bf8d6cccfbb038892eca3244f7521be43d812bcb1720824f0fc","title":"Clearing export_hashes doesn't trigger full re-export, causing database divergence","description":"## Problem\n\nWhen `bd sync` detects JSONL file hash mismatch (bd-160 warning) and clears `export_hashes` to \"force full re-export\", the subsequent export **doesn't actually re-export all issues**. This causes databases to diverge permanently.\n\n## Evidence\n\nComparing /Users/stevey/src/fred/beads and ~/src/beads after both ran sync:\n- Fred has 204 lines in JSONL, src has 199 lines\n- 5 issues exist in Fred's DB and JSONL but missing from src's JSONL:\n - bd-47c59dd4, bd-cb64c226.2, bd-cb64c226.3, bd-ef1a1c26 (all closed)\n - bd-eff3 (open)\n- Fred's `export_hashes` table is EMPTY (0 rows)\n- Sync cleared export_hashes but didn't rebuild it\n\n## Root Cause\n\nautoflush.go lines 427-436:\n1. Detects JSONL hash mismatch\n2. Clears `export_hashes` table to \"force full re-export\"\n3. But export logic then only exports issues with changes\n4. Since export_hashes is empty, it has no baseline - should export ALL issues\n5. **BUG**: Export doesn't treat empty export_hashes as \"export everything\"\n6. Result: Issues in DB but not in export_hashes tracking are silently skipped\n\n## Impact\n\n- Critical: Causes permanent database divergence between repos\n- Multiple agents working on same project see different issue lists\n- Silent data loss - no error shown, issues just disappear from sync\n- Breaking fundamental git-as-source-of-truth model\n\n## Reproduction\n\n1. Two clones of same repo (fred and src)\n2. Fred runs sync, triggers bd-160 warning (JSONL hash mismatch)\n3. export_hashes gets cleared\n4. Subsequent exports skip issues not in (empty) export_hashes\n5. Fred and src databases permanently diverge\n\n## Fix Needed\n\nWhen `export_hashes` is cleared, the next export must:\n1. Detect that export_hashes is empty (or count rows)\n2. Export ALL issues from database\n3. Rebuild export_hashes from scratch\n4. OR: Immediately trigger full re-export after clearing export_hashes\n\nAlternatively: Don't clear export_hashes - find root cause of hash mismatch and fix that instead.","notes":"## Fix Implemented (v2 - Improved after code review)\n\n### Changes to validateJSONLIntegrity()\n- Returns `(needsFullExport bool, error)` instead of just `error`\n- Returns `needsFullExport=true` when export_hashes is cleared (hash mismatch or missing JSONL)\n\n### Changes to flushToJSONL()\n**Critical control flow fixes:**\n1. **Moved integrity check BEFORE isDirty gate** - integrity issues now trigger export even when nothing is dirty\n2. **Treat missing JSONL as non-fatal** - forces full export instead of failing\n3. **Combined dirty and integrity checks** - proceeds if either isDirty OR integrityNeedsFullExport\n4. **Clear flags only after deciding to proceed** - prevents losing dirty state on validation errors\n\n**Scanner buffer improvements:**\n5. Increased buffer size from 64KB to 2MB to handle large JSON lines\n6. Added scanner.Err() check to catch buffer overflow errors\n\n### Test Updates\nUpdated all test files to handle new validateJSONLIntegrity signature and verify needsFullExport flag.\n\n### Key Improvements Over v1\n- **Handles \"not dirty but needs full export\" case** - integrity issues trigger export independently\n- **Missing JSONL no longer blocks export** - treated as \"force full export\" condition\n- **Large lines don't cause silent data loss** - increased scanner buffer and error checking\n- **Validation errors don't clear isDirty** - failed validation won't lose pending changes\n\n### Tests Confirm\n- When JSONL hash mismatches, export_hashes is cleared AND full export occurs (even if not dirty)\n- When JSONL is missing, export_hashes is cleared AND JSONL is recreated\n- Large JSON lines (\u003e64KB) are handled correctly without truncation\n- All integrity validation and autoflush tests pass","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-11-01T20:17:35.532549-07:00","updated_at":"2025-11-01T20:24:31.540439-07:00","closed_at":"2025-11-01T20:21:51.722246-07:00"}
{"id":"bd-c796","content_hash":"25587342dd6e95a6435e01a4d952b3639f29cc088f62a480761a4d7832cae5ab","title":"Extract batch operations to batch_ops.go","description":"Move validateBatchIssues, generateBatchIDs, bulkInsertIssues, bulkRecordEvents, bulkMarkDirty, CreateIssues to batch_ops.go","status":"closed","priority":1,"issue_type":"task","created_at":"2025-11-01T19:28:54.887487-07:00","updated_at":"2025-11-01T19:53:42.93284-07:00","closed_at":"2025-11-01T19:53:42.93284-07:00"}
{"id":"bd-c825f867","content_hash":"27cecaa2dc6cdabb2ae77fd65fbf8dca8f4c536bdf140a13b25cdd16376c9845","title":"Add docs/architecture/event_driven.md","description":"Copy event_driven_daemon.md into docs/ folder. Add to documentation index.","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-28T16:20:02.431399-07:00","updated_at":"2025-10-30T17:12:58.221939-07:00"}
{"id":"bd-c947dd1b","content_hash":"79bd51b46b28bc16cfc19cd19a4dd4f57f45cd1e902b682788d355b03ec00b2a","title":"Remove Daemon Storage Cache","description":"The daemon's multi-repo storage cache is the root cause of stale data bugs. Since global daemon is deprecated, we only ever serve one repository, making the cache unnecessary complexity. This epic removes the cache entirely for simpler, more reliable direct storage access.","design":"For local daemon (single repository), eliminate the cache entirely:\n- Use s.storage field directly (opened at daemon startup)\n- Remove getStorageForRequest() routing logic\n- Remove server_cache_storage.go entirely (~300 lines)\n- Remove cache-related tests\n- Simplify Server struct\n\nBenefits:\n✅ No staleness bugs: Always using live SQLite connection\n✅ Simpler code: Remove ~300 lines of cache management\n✅ Easier debugging: Direct storage access, no cache indirection\n✅ Same performance: Cache was always 1 entry for local daemon anyway","acceptance_criteria":"- Daemon has no storage cache code\n- All tests pass\n- MCP integration works\n- No stale data bugs\n- Documentation updated\n- Performance validated","status":"closed","priority":1,"issue_type":"epic","created_at":"2025-10-28T10:50:15.126939-07:00","updated_at":"2025-10-30T17:12:58.21743-07:00","closed_at":"2025-10-28T10:49:53.612049-07:00"}