johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

bd sync: 2025-12-24 23:12:58

Browse Source
This commit is contained in:
Steve Yegge
2025-12-24 23:12:58 -08:00
parent a1a4341b41
commit fecbc5be76
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.beads/issues.jsonl
View File

@@ -193,6 +193,7 @@
{"id":"bd-c3u","title":"Review PR #512: clarify bd ready docs","description":"Review and merge PR #512 from aspiers. This PR clarifies what bd ready does after git pull in README.md. Simple 1-line change. URL: https://github.com/anthropics/beads/pull/512","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-13T08:15:13.405161+11:00","updated_at":"2025-12-13T07:07:29.641265-08:00","closed_at":"2025-12-13T07:07:29.641265-08:00"}
{"id":"bd-c7y5","title":"Optimization: Tombstones still synced, adding overhead","description":"Tombstoned (deleted) issues are still processed during sync, adding overhead.\n\n## Evidence\n```\nImport complete: 0 created, 0 updated, 407 unchanged, 100 skipped\n```\nThose 100 skipped are tombstones - they're read from JSONL, parsed, then filtered out.\n\n## Current State (Beads repo)\n- 408 total issues\n- 99 tombstones (24% of database)\n- Every sync reads and skips these 99 entries\n\n## Impact\n- Sync time increases with tombstone count\n- JSONL file size grows indefinitely\n- Git history accumulates tombstone churn\n\n## Proposed Solutions\n\n### 1. JSONL Compaction (`bd compact`)\nPeriodically rewrite JSONL without tombstones:\n```bash\nbd compact # Removes tombstones, rewrites issues.jsonl\n```\nTrade-off: Loses delete history, but that's in git anyway.\n\n### 2. Tombstone TTL\nAuto-remove tombstones older than N days during sync:\n```go\nif issue.Deleted \u0026\u0026 time.Since(issue.UpdatedAt) \u003e 7*24*time.Hour {\n // Skip writing to new JSONL\n}\n```\n\n### 3. Archive File\nMove old closed issues to `issues-archive.jsonl`:\n- Not synced regularly\n- Available for historical queries\n- Main JSONL stays small\n\n### 4. Lazy Tombstone Handling \nDon't write tombstones to JSONL at all - just remove the line:\n- Simpler, but loses cross-clone delete propagation\n- Would need different delete propagation mechanism\n\n## Recommendation\nStart with `bd compact` command - simple, explicit, user-controlled.\n\n## Related\n- gt-tnss: Analysis - Beads database size and hygiene strategy\n- gt-ox67: Maintenance - Regular cleanup of closed MR/gate beads","status":"closed","priority":2,"issue_type":"task","assignee":"beads/dag","created_at":"2025-12-23T14:41:16.925212-08:00","updated_at":"2025-12-23T23:50:23.363222-08:00","closed_at":"2025-12-23T23:50:23.363222-08:00","close_reason":"Added bd compact --prune mode for standalone tombstone pruning"}
{"id":"bd-c83r","title":"Prevent multiple daemons from running on the same repo","description":"Multiple bd daemons running on the same repo clone causes race conditions and data corruption risks.\n\n**Problem:**\n- Nothing prevents spawning multiple daemons for the same repository\n- Multiple daemons watching the same files can conflict during sync operations\n- Observed: 4 daemons running simultaneously caused sync race condition\n\n**Solution:**\nImplement daemon singleton enforcement per repo:\n1. Use a lock file (e.g., .beads/.daemon.lock) with PID\n2. On daemon start, check if lock exists and process is alive\n3. If stale lock (dead PID), clean up and acquire lock\n4. If active daemon exists, either:\n - Exit with message 'daemon already running (PID xxx)'\n - Or offer --replace flag to kill existing and take over\n5. Release lock on graceful shutdown\n\n**Edge cases to handle:**\n- Daemon crashes without releasing lock (stale PID detection)\n- Multiple repos in same directory tree (each repo gets own lock)\n- Race between two daemons starting simultaneously (atomic lock acquisition)","status":"closed","priority":2,"issue_type":"bug","created_at":"2025-12-13T06:37:23.377131-08:00","updated_at":"2025-12-16T01:14:49.50347-08:00","closed_at":"2025-12-14T17:34:14.990077-08:00"}
{"id":"bd-c8d5","title":"bd mol run: Only creates partial children from proto","description":"When running `bd mol run gt-lwuu --var issue=gt-xxx`, only 2 of 4 proto children are created.\n\n## Expected\nProto gt-lwuu has 4 children:\n- gt-lwuu.1: load-context\n- gt-lwuu.2: implement \n- gt-lwuu.3: self-review\n- gt-lwuu.8: request-shutdown\n\nAll 4 should be instantiated.\n\n## Actual\nOnly 2 children created:\n- load-context\n- implement\n\nself-review and request-shutdown are missing.\n\n## Repro\n```bash\nbd --no-daemon mol run gt-lwuu --var issue=gt-xxx --json\nbd list --parent=\u003ccreated-id\u003e # Only shows 2 children\n```\n\n## Impact\nPolecats spawned with mol-polecat-work get incomplete workflow.","status":"open","priority":1,"issue_type":"bug","assignee":"beads/dave","created_at":"2025-12-24T23:12:47.923073-08:00","updated_at":"2025-12-24T23:12:58.457595-08:00"}
{"id":"bd-cb64c226.1","title":"Performance Validation","description":"Confirm no performance regression from cache removal","status":"tombstone","priority":1,"issue_type":"task","created_at":"2025-10-28T10:50:15.126019-07:00","updated_at":"2025-12-17T23:18:29.108883-08:00","close_reason":"Closed","deleted_at":"2025-12-17T23:18:29.108883-08:00","deleted_by":"daemon","delete_reason":"delete","original_type":"task"}
{"id":"bd-cb64c226.10","title":"Delete server_cache_storage.go","description":"Remove the entire cache implementation file (~286 lines)","status":"tombstone","priority":1,"issue_type":"task","created_at":"2025-10-27T22:55:38.729299-07:00","updated_at":"2025-12-17T23:18:29.110716-08:00","close_reason":"Closed","deleted_at":"2025-12-17T23:18:29.110716-08:00","deleted_by":"daemon","delete_reason":"delete","original_type":"task"}
{"id":"bd-cb64c226.12","title":"Remove Storage Cache from Server Struct","description":"Eliminate cache fields and use s.storage directly","status":"tombstone","priority":1,"issue_type":"task","created_at":"2025-10-27T22:55:25.474412-07:00","updated_at":"2025-12-17T23:18:29.111039-08:00","close_reason":"Closed","deleted_at":"2025-12-17T23:18:29.111039-08:00","deleted_by":"daemon","delete_reason":"delete","original_type":"task"}
@@ -221,6 +222,7 @@
{"id":"bd-dhza","title":"Reduce global state in cmd/bd/main.go (25+ variables)","description":"Code health review found main.go has 25+ global variables (lines 57-112):\n\n- dbPath, actor, store, jsonOutput, daemonClient, noDaemon\n- rootCtx, rootCancel, autoFlushEnabled\n- isDirty (marked 'USED BY LEGACY CODE')\n- needsFullExport (marked 'USED BY LEGACY CODE')\n- flushTimer (marked 'DEPRECATED')\n- flushMutex, storeMutex, storeActive\n- flushFailureCount, lastFlushError, flushManager\n- skipFinalFlush, autoImportEnabled\n- versionUpgradeDetected, previousVersion, upgradeAcknowledged\n\nImpact:\n- Hard to test individual commands\n- Race conditions possible\n- State leakage between commands\n\nFix: Move toward dependency injection. Remove deprecated variables. Consider cmd/bd/internal package.","notes":"COMPLETED: Migration of legacy flush state to FlushManager. The main work (isDirty, needsFullExport, flushTimer) moved to local vars in FlushManager.run() lines 170-173. This fixes the race conditions. Remaining globals are: Cobra flag bindings (standard), runtime state (store, daemonClient - structural), store coordination (storeActive/storeMutex). Future: move error tracking to FlushManager, create cmd/bd/internal for encapsulation.","status":"closed","priority":3,"issue_type":"task","assignee":"beads/kilo","created_at":"2025-12-16T18:17:29.643293-08:00","updated_at":"2025-12-23T22:36:09.600594-08:00","closed_at":"2025-12-23T22:36:09.600594-08:00","close_reason":"Legacy flush globals (isDirty, needsFullExport, flushTimer) already migrated to FlushManager local vars. Race conditions fixed. Remaining globals are Cobra flags (standard) or structural (store, daemonClient). Future work filed separately if needed."}
{"id":"bd-dp4w","title":"Test message","description":"This is a test message body","status":"tombstone","priority":2,"issue_type":"message","created_at":"2025-12-16T18:11:58.467876-08:00","updated_at":"2025-12-17T16:11:17.070763-08:00","deleted_at":"2025-12-17T16:11:17.070763-08:00","deleted_by":"batch delete","delete_reason":"batch delete","original_type":"message"}
{"id":"bd-dqu8","title":"Restart running daemons","description":"Kill and restart any running bd daemons to pick up new version: pkill -f 'bd daemon' \u0026\u0026 bd daemon --start","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-20T00:32:26.559311-08:00","updated_at":"2025-12-20T00:32:59.123766-08:00","closed_at":"2025-12-20T00:32:59.123766-08:00","close_reason":"Daemons restarted - now running 0.30.7","dependencies":[{"issue_id":"bd-dqu8","depends_on_id":"bd-6s61","type":"parent-child","created_at":"2025-12-20T00:32:39.36213-08:00","created_by":"daemon"},{"issue_id":"bd-dqu8","depends_on_id":"bd-fgw3","type":"blocks","created_at":"2025-12-20T00:32:39.427846-08:00","created_by":"daemon"}]}
{"id":"bd-drcx","title":"bd mol run: Support proto lookup by title","description":"`bd mol run` only accepts proto ID (e.g., gt-lwuu), not proto title (e.g., mol-polecat-work).\n\n## Current Behavior\n```bash\nbd mol run mol-polecat-work --var issue=gt-xxx\n# Error: no issue found matching \"mol-polecat-work\"\n\nbd mol run gt-lwuu --var issue=gt-xxx \n# Works\n```\n\n## Desired Behavior\nBoth should work - lookup by ID or by title.\n\n## Why This Matters\n- Proto titles are human-readable and memorable\n- Proto IDs are opaque (gt-lwuu vs mol-polecat-work)\n- Hardcoding IDs in code is fragile (IDs change across databases)\n- `bd mol catalog` shows both: `gt-lwuu: mol-polecat-work`\n\n## Workaround\nCurrently using hardcoded proto ID in gt spawn, which is brittle.","status":"open","priority":2,"issue_type":"feature","assignee":"beads/dave","created_at":"2025-12-24T23:12:49.151242-08:00","updated_at":"2025-12-24T23:12:58.531468-08:00"}
{"id":"bd-drxs","title":"Make merge requests ephemeral wisps instead of permanent issues","description":"## Problem\n\nMerge requests (MRs) are currently created as regular beads issues (type: merge-request). This means they:\n- Sync to JSONL and propagate via git\n- Accumulate in the issue database indefinitely\n- Clutter `bd list` output with closed MRs\n- Create permanent records for inherently transient artifacts\n\nMRs are process artifacts, not work products. They exist briefly while code awaits merge, then their purpose is fulfilled. The git merge commit and GitHub PR (if applicable) provide the permanent audit trail - the beads MR is redundant.\n\n## Proposed Solution\n\nMake MRs ephemeral wisps that exist only during the merge process:\n\n1. **Create MRs as wisps**: When a polecat completes work and requests merge, create the MR in `.beads-wisp/` instead of `.beads/`\n\n2. **Refinery visibility**: This works because all clones within a rig share the same database:\n ```\n beads/ ← Rig root\n ├── .beads/ ← Permanent issues (synced to JSONL)\n ├── .beads-wisp/ ← Ephemeral wisps (NOT synced)\n ├── crew/dave/ ← Uses rig's shared DB\n ├── polecats/*/ ← Uses rig's shared DB\n └── refinery/ ← Uses rig's shared DB\n ```\n The refinery can see wisp MRs immediately - same SQLite database.\n\n3. **On merge completion**: Burn the wisp (delete without digest). The git merge commit IS the permanent record. No digest needed since:\n - Digest wouldn't be smaller than the MR itself (~200-300 bytes either way)\n - Git history provides complete audit trail\n - GitHub PR (if used) provides discussion/approval record\n\n4. **On merge rejection/abandonment**: Burn the wisp. Optionally notify the source polecat via mail.\n\n## Benefits\n\n- **Clean JSONL**: MRs never pollute the permanent issue history\n- **No accumulation**: Wisps are burned on completion, no cleanup needed\n- **Correct semantics**: Wisps are for \"operational ephemera\" - MRs fit perfectly\n- **Reduced sync churn**: Fewer JSONL updates, faster `bd sync`\n- **Cleaner queries**: `bd list` shows work items, not process artifacts\n\n## Implementation Notes\n\n### Where MRs are created\n\nCurrently MRs are created by the witness or polecat when work is ready for merge. This code needs to:\n- Set `wisp: true` on the MR issue\n- Or use a dedicated wisp creation path\n\n### Refinery changes\n\nThe refinery queries for pending MRs to process. It needs to:\n- Query wisp storage as well as (or instead of) permanent storage\n- Use `bd mol burn` or equivalent to delete processed MRs\n\n### What about cross-rig MRs?\n\nIf an MR needs to be visible outside the rig (e.g., external collaborators):\n- They would see the GitHub PR anyway\n- Or we could create a permanent \"merge completed\" notification issue\n- But this is likely unnecessary - MRs are internal coordination\n\n### Migration\n\nExisting MRs in permanent storage:\n- Can be cleaned up with `bd cleanup` or manual deletion\n- Or left to age out naturally\n- No migration of open MRs needed (they'll complete under old system\n\n## Alternatives Considered\n\n1. **Auto-cleanup of closed MRs**: Keep MRs as permanent issues but auto-delete after 24h. Simpler but still creates sync churn and temporary JSONL pollution.\n\n2. **MRs as mail only**: Polecat sends mail to refinery with merge details, no MR issue at all. Loses queryability (bd-801b [P2] [merge-request] closed - Merge: bd-bqcc\nbd-pvu0 [P2] [merge-request] closed - Merge: bd-4opy\nbd-i0rx [P2] [merge-request] closed - Merge: bd-ao0s\nbd-u0sb [P2] [merge-request] closed - Merge: bd-uqfn\nbd-8e0q [P2] [merge-request] closed - Merge: beads-ocs\nbd-hvng [P2] [merge-request] closed - Merge: bd-w193\nbd-4sfl [P2] [merge-request] closed - Merge: bd-14ie\nbd-sumr [P2] [merge-request] closed - Merge: bd-t4sb\nbd-3x9o [P2] [merge-request] closed - Merge: bd-by0d\nbd-whgv [P2] [merge-request] closed - Merge: bd-401h\nbd-f3ll [P2] [merge-request] closed - Merge: bd-ot0w\nbd-fmdy [P3] [merge-request] closed - Merge: bd-kzda).\n\n3. **Separate merge queue**: Refinery maintains internal state for pending merges, not in beads at all. Clean but requires new infrastructure.\n\nWisps are the cleanest solution - they already exist, have the right semantics, and require minimal changes.\n\n## Related\n\n- Wisp architecture: \n- Current MR creation: witness/refinery code paths\n- bd-pvu0, bd-801b: Example MRs currently in permanent storage\nEOF\n)","status":"tombstone","priority":0,"issue_type":"feature","created_at":"2025-12-23T01:39:25.4918-08:00","updated_at":"2025-12-23T01:58:23.550668-08:00","deleted_at":"2025-12-23T01:58:23.550668-08:00","deleted_by":"daemon","delete_reason":"delete","original_type":"feature"}
{"id":"bd-dsdh","title":"Document sync.branch 'always dirty' working tree behavior","description":"## Context\n\nWhen sync.branch is configured, the .beads/issues.jsonl file in main's working tree is ALWAYS dirty. This is by design:\n\n1. bd sync commits to beads-sync branch (via worktree)\n2. bd sync copies JSONL to main's working tree (so CLI commands work)\n3. This copy is NOT committed to main (to reduce commit noise)\n\nContributors who watch main branch history pushed for sync.branch to avoid constant beads commit noise. But users need to understand the trade-off.\n\n## Documentation Needed\n\nUpdate README.md sync.branch section with:\n\n1. **Clear explanation** of why .beads/ is always dirty on main\n2. **\"Be Zen about it\"** - this is expected, not a bug\n3. **Workflow options:**\n - Accept dirty state, use `bd sync --merge` periodically to snapshot to main\n - Or disable sync.branch if clean working tree is more important\n4. **Shell alias tip** to hide beads from git status:\n ```bash\n alias gs='git status -- \":!.beads/\"'\n ```\n5. **When to merge**: releases, milestones, or periodic snapshots\n\n## Related\n\n- bd-7b7h: Fix that allows bd sync --merge to work with dirty .beads/\n- bd-elqd: Investigation that identified this as expected behavior","status":"tombstone","priority":2,"issue_type":"task","created_at":"2025-12-16T23:16:12.253559-08:00","updated_at":"2025-12-17T16:11:17.070763-08:00","deleted_at":"2025-12-17T16:11:17.070763-08:00","deleted_by":"batch delete","delete_reason":"batch delete","original_type":"task"}
{"id":"bd-dsp","title":"Test stdin body-file","status":"closed","priority":4,"issue_type":"task","created_at":"2025-12-17T17:27:32.098806-08:00","updated_at":"2025-12-17T17:28:33.832749-08:00","closed_at":"2025-12-17T17:28:33.832749-08:00"}