johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(doctor): add tombstone health checks (bd-s3v)

Browse Source 
Add two new doctor checks for tombstone health:

1. Updated Deletions Manifest check:
   - Warns when legacy deletions.jsonl has entries (suggests migration)
   - Shows "Migrated to tombstones" when .migrated file exists
   - Shows "Using inline tombstones" for new repos

2. New Tombstones check:
   - Reports total tombstone count
   - Warns about expired tombstones (older than 30 days)
   - Shows tombstones expiring within 7 days
   - Suggests 'bd compact' to prune expired tombstones

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Steve Yegge
2025-12-13 10:16:27 -08:00
parent 2c6748fd59
commit a0f4a9cacd
2 changed files with 130 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.beads/issues.jsonl
View File

@@ -386,7 +386,6 @@
{"id":"bd-hm8","title":"Add uninstall documentation (GH #445)","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-01T21:04:30.313637-08:00","updated_at":"2025-12-01T21:04:57.943192-08:00","closed_at":"2025-12-01T21:04:57.943192-08:00"}
{"id":"bd-ho5","title":"Add 'town report' command for aggregated swarm status","description":"## Problem\nGetting a full swarm status requires running 6+ commands:\n- `town list \u003crig\u003e` for each rig\n- `town mail inbox` as Boss\n- `bd list --status=open/in_progress` per rig\n\nThis is slow and error-prone for both humans and agents.\n\n## Proposed Solution\nAdd `town report [RIG]` command that aggregates:\n- All rigs with polecat states (running/stopped, awake/asleep)\n- Boss inbox summary (unread count, recent senders)\n- Aggregate issue counts per rig (open/in_progress/blocked)\n\nExample output:\n```\n=== beads ===\nPolecats: 5 (5 running, 0 stopped)\nIssues: 20 open, 0 in_progress, 0 blocked\n\n=== gastown ===\nPolecats: 6 (4 running, 2 stopped)\nIssues: 0 open, 0 in_progress, 0 blocked\n\n=== Boss Mail ===\nUnread: 10 | Total: 22\nRecent: rictus (21:19), scrotus (21:14), immortanjoe (21:14)\n```\n\n## Acceptance Criteria\n- [ ] `town report` shows all rigs\n- [ ] `town report \u003crig\u003e` shows single rig detail\n- [ ] Output is concise and scannable\n- [ ] Completes in \u003c2 seconds","status":"closed","priority":2,"issue_type":"feature","created_at":"2025-11-27T22:55:36.8919-08:00","updated_at":"2025-11-27T22:56:08.071838-08:00","closed_at":"2025-11-27T22:56:08.071838-08:00"}
{"id":"bd-hp0m","title":"Add test for legacy deletions.jsonl to tombstone conversion","description":"The importer now converts legacy deletions.jsonl entries to tombstones (bd-dve), but there's no dedicated test that:\n1. Creates a deletions.jsonl with entries\n2. Imports issues (some in deletions, some not)\n3. Verifies the converted tombstones have correct fields from the deletion record\n\nThe existing TestImportIssues_TombstoneNotFilteredByDeletionsManifest tests that tombstones bypass the filter, but doesn't verify the conversion of legacy deletions to tombstones.\n\nAdd a test: TestImportIssues_LegacyDeletionsConvertedToTombstones","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T01:41:12.976726-08:00","updated_at":"2025-12-07T02:16:16.882286-08:00","closed_at":"2025-12-07T02:16:16.882286-08:00","dependencies":[{"issue_id":"bd-hp0m","depends_on_id":"bd-dve","type":"blocks","created_at":"2025-12-07T01:41:28.391366-08:00","created_by":"daemon"}]}
{"id":"bd-hxou","title":"Daemon performAutoImport should update jsonl_content_hash after import","description":"The daemon's performAutoImport function was not updating jsonl_content_hash after successful import, unlike the CLI import path. This could cause repeated imports if the file hash and DB hash diverge.\n\nFix: Added hash update after validatePostImport succeeds, using repoKey for multi-repo support.","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-13T06:44:04.442976-08:00","updated_at":"2025-12-13T06:44:09.546976-08:00","closed_at":"2025-12-13T06:44:09.546976-08:00"}
{"id":"bd-ig5","title":"Duplicate tombstone constants in merge and types packages","description":"## Problem\n\nThe tombstone constants are duplicated between two packages:\n\n**internal/types/types.go:**\n```go\nconst DefaultTombstoneTTL = 30 * 24 * time.Hour\nconst ClockSkewGrace = 1 * time.Hour\nconst StatusTombstone Status = \"tombstone\"\n```\n\n**internal/merge/merge.go:**\n```go\nconst StatusTombstone = \"tombstone\"\nconst DefaultTombstoneTTL = 30 * 24 * time.Hour\nconst ClockSkewGrace = 1 * time.Hour\n```\n\nThis violates DRY and creates risk of divergence if one is updated but not the other.\n\n## Root Cause\n\nThe merge package has its own `Issue` struct (for JSONL parsing) and cannot import types.Issue directly due to the different struct design. However, the constants could be shared.\n\n## Recommendation\n\nExport the constants from types package and import them in merge:\n\n```go\n// merge/merge.go\nimport \"github.com/steveyegge/beads/internal/types\"\n\n// Use types.StatusTombstone, types.DefaultTombstoneTTL, types.ClockSkewGrace\n```\n\nOr create a shared constants package if import cycles are a concern.\n\n## Files\n- internal/merge/merge.go:241-248\n- internal/types/types.go:77-84, 170","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-05T16:36:06.159443-08:00","updated_at":"2025-12-05T17:15:57.624376-08:00","closed_at":"2025-12-05T17:15:57.624376-08:00"}
{"id":"bd-iip5","title":"TestImportIssues_LegacyDeletionsConvertedToTombstones is failing","description":"The test TestImportIssues_LegacyDeletionsConvertedToTombstones in internal/importer/importer_test.go is failing:\n\nExpected 3 created (1 regular + 2 tombstones from deletions.jsonl), got 2\nExpected tombstone for test-deleted2 not found\n\nThe test expects legacy deletions.jsonl entries to be converted to tombstones during import, but test-deleted2 is not being converted.\n\nLocation: internal/importer/importer_test.go:1344","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-07T02:09:19.17774-08:00","updated_at":"2025-12-07T02:16:55.422522-08:00","closed_at":"2025-12-07T02:16:55.422522-08:00"}
{"id":"bd-imj","title":"Deletion propagation via deletions manifest","description":"## Problem\n\nWhen `bd cleanup -f` or `bd delete` removes issues in one clone, those deletions don't propagate to other clones. The import logic only creates/updates, never deletes. This causes \"resurrection\" where deleted issues reappear.\n\n## Root Cause\n\nImport sees DB issues not in JSONL and assumes they're \"local unpushed work\" rather than \"intentionally deleted upstream.\"\n\n## Solution: Deletions Manifest\n\nAdd `.beads/deletions.jsonl` - an append-only log of deleted issue IDs with metadata.\n\n### Format\n```jsonl\n{\"id\":\"bd-xxx\",\"ts\":\"2025-11-25T10:00:00Z\",\"by\":\"stevey\"}\n{\"id\":\"bd-yyy\",\"ts\":\"2025-11-25T10:05:00Z\",\"by\":\"claude\",\"reason\":\"duplicate of bd-zzz\"}\n```\n\n### Fields\n- `id`: Issue ID (required)\n- `ts`: ISO 8601 UTC timestamp (required)\n- `by`: Actor who deleted (required)\n- `reason`: Optional context (\"cleanup\", \"duplicate of X\", etc.)\n\n### Import Logic\n```\nFor each DB issue not in JSONL:\n 1. Check deletions manifest → if found, delete from DB\n 2. Fallback: check git history → if found, delete + backfill manifest\n 3. Neither → keep (local unpushed work)\n```\n\n### Conflict Resolution\nSimultaneous deletions from multiple clones are handled naturally:\n- Append-only design means both clones append their deletion records\n- On merge, file contains duplicate entries (same ID, different timestamps)\n- `LoadDeletions` deduplicates by ID (keeps any/first entry)\n- Result: deletion propagates correctly, duplicates are harmless\n\n### Pruning Policy\n- Default retention: 7 days (configurable via `deletions.retention_days`)\n- Auto-compact during `bd sync` is **opt-in** (disabled by default)\n- Hard cap: `deletions.max_entries` (default 50000)\n- Git fallback handles pruned entries (self-healing)\n\n### Self-Healing\nWhen git fallback catches a resurrection (pruned entry), it backfills the manifest. One-time git scan cost, then fast again.\n\n### Size Estimates\n- ~80 bytes/entry\n- 7-day retention with 100 deletions/day = ~56KB\n- Git compressed: ~10KB\n\n## Benefits\n- ✅ Deletions propagate across clones\n- ✅ O(1) lookup (no git scan in normal case)\n- ✅ Works in shallow clones\n- ✅ Survives history rewrite\n- ✅ Audit trail (who deleted what when)\n- ✅ Self-healing via git fallback\n- ✅ Bounded size via time-based pruning\n\n## References\n- Investigation session: 2025-11-25\n- Related: bd-2q6d (stale database warnings)","status":"closed","priority":0,"issue_type":"epic","created_at":"2025-11-25T09:56:01.98027-08:00","updated_at":"2025-11-25T16:36:27.965168-08:00","closed_at":"2025-11-25T16:36:27.965168-08:00","dependencies":[{"issue_id":"bd-imj","depends_on_id":"bd-qsm","type":"blocks","created_at":"2025-11-25T09:57:42.821911-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-x2i","type":"blocks","created_at":"2025-11-25T09:57:42.851712-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-44e","type":"blocks","created_at":"2025-11-25T09:57:42.88154-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-bhd","type":"blocks","created_at":"2025-11-25T14:56:23.675787-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-bgs","type":"blocks","created_at":"2025-11-25T14:56:23.744648-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-f0n","type":"blocks","created_at":"2025-11-25T14:56:23.80649-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-v29","type":"blocks","created_at":"2025-11-25T14:56:23.864569-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-mdw","type":"blocks","created_at":"2025-11-25T14:56:48.592492-08:00","created_by":"daemon"},{"issue_id":"bd-imj","depends_on_id":"bd-03r","type":"blocks","created_at":"2025-11-25T14:56:54.295851-08:00","created_by":"daemon"}]}
@@ -427,7 +426,7 @@
{"id":"bd-r9iq","title":"purgeDeletedIssues still writes to deletions.jsonl during git-history-backfill","description":"In purgeDeletedIssues (lines 929-939), when a deletion is recovered from git history, we:\n1. Append to deletions.jsonl (backfill)\n2. Create a tombstone in DB\n\nWith the tombstone approach, writing to deletions.jsonl is redundant since the tombstone will be exported to JSONL. Consider removing the deletions.jsonl backfill once we're confident in the tombstone approach.\n\nThis is related to the Phase 2 migration (bd-vw8) - we should stop writing to deletions.jsonl entirely.","status":"open","priority":2,"issue_type":"task","created_at":"2025-12-07T01:40:42.581876-08:00","updated_at":"2025-12-07T01:40:42.581876-08:00","dependencies":[{"issue_id":"bd-r9iq","depends_on_id":"bd-dve","type":"blocks","created_at":"2025-12-07T01:41:28.285457-08:00","created_by":"daemon"}]}
{"id":"bd-s0z","title":"Consider extracting error handling helpers","description":"Evaluate creating FatalError() and WarnError() helpers as suggested in ERROR_HANDLING.md to reduce boilerplate and enforce consistency. Prototype in a few files first to validate the approach.","status":"closed","priority":4,"issue_type":"task","created_at":"2025-11-24T00:28:57.248959-08:00","updated_at":"2025-11-30T10:50:04.275143-08:00","closed_at":"2025-11-28T23:28:00.886536-08:00"}
{"id":"bd-s2t","title":"wish: a 'continue' or similar cmd/flag which means alter last issue","description":"so many time I create an issue and then have another thought: 'oh, before I did X and it crashed there was ZZZ happening' or 'actually that is P4 not P2'. It would be nice if when `bd {cmd}` is used without a {title} or {id} it just adds or updates the most recently touched issue.","status":"open","priority":3,"issue_type":"task","created_at":"2025-12-08T06:46:37.529160416-07:00","updated_at":"2025-12-08T06:46:37.529160416-07:00"}
{"id":"bd-s3v","title":"Add tombstone doctor checks","description":"Add bd doctor checks for: unmigrated deletions.jsonl, stale deletions.jsonl in Phase 3, tombstones expiring soon. Per design bd-dli.","status":"open","priority":2,"issue_type":"task","created_at":"2025-12-05T15:14:46.733558-08:00","updated_at":"2025-12-05T15:14:46.733558-08:00","dependencies":[{"issue_id":"bd-s3v","depends_on_id":"bd-8f9","type":"blocks","created_at":"2025-12-05T15:14:59.006029-08:00","created_by":"daemon"}]}
{"id":"bd-s3v","title":"Add tombstone doctor checks","description":"Add bd doctor checks for: unmigrated deletions.jsonl, stale deletions.jsonl in Phase 3, tombstones expiring soon. Per design bd-dli.","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:14:46.733558-08:00","updated_at":"2025-12-13T10:15:38.981577-08:00","closed_at":"2025-12-13T10:15:38.981577-08:00","dependencies":[{"issue_id":"bd-s3v","depends_on_id":"bd-8f9","type":"blocks","created_at":"2025-12-05T15:14:59.006029-08:00","created_by":"daemon"}]}
{"id":"bd-saa","title":"Add index on deleted_at for TTL queries","description":"Per bd-2m7 design: 'Index for efficient tombstone filtering' was recommended. The current implementation does NOT add an index on deleted_at.\n\nFor TTL cleanup queries like 'SELECT * FROM issues WHERE deleted_at \u003c datetime(now, -30 days)', this will require a full table scan.\n\nAdd to migration or new migration:\nCREATE INDEX IF NOT EXISTS idx_issues_deleted_at ON issues(deleted_at) WHERE deleted_at IS NOT NULL","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-05T15:35:33.842664-08:00","updated_at":"2025-12-05T15:48:07.584268-08:00","closed_at":"2025-12-05T15:48:07.584268-08:00","dependencies":[{"issue_id":"bd-saa","depends_on_id":"bd-vw8","type":"parent-child","created_at":"2025-12-05T15:35:47.222711-08:00","created_by":"daemon"}]}
{"id":"bd-t3b","title":"Add test coverage for internal/config package","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-11-20T21:21:22.91657-05:00","updated_at":"2025-11-30T10:50:04.275382-08:00","closed_at":"2025-11-28T21:54:15.009889-08:00","dependencies":[{"issue_id":"bd-t3b","depends_on_id":"bd-ge7","type":"blocks","created_at":"2025-11-20T21:21:31.201036-05:00","created_by":"daemon"}]}
{"id":"bd-t5m","title":"CRITICAL: git-history-backfill purges entire database when JSONL reset","description":"When a clone gets reset (git reset --hard origin/main), the git-history-backfill logic incorrectly adds ALL issues to the deletions manifest, then sync purges the entire database.\\n\\nFix adds safety guard: never delete more than 50% of issues via git-history-backfill. If threshold exceeded, abort with warning message.","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-11-30T21:24:47.397394-08:00","updated_at":"2025-11-30T21:24:52.710971-08:00","closed_at":"2025-11-30T21:24:52.710971-08:00"}