From 4f481c88b1cee436f896785378c09b8e7f399781 Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Thu, 20 Nov 2025 19:27:41 -0500 Subject: [PATCH] Close bd-5qim (GetReadyWork optimization completed in 690c73f) --- .beads/issues.jsonl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 2d741e29..be969872 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -2166,7 +2166,7 @@ {"id":"bd-5ki8","content_hash":"d89e5e528819934bcb7ee162fa7e32c27298db5816ecf51bcc8ede1809f1d5b9","title":"Add integration tests for adapter library","description":"Test suite for beads_mail_adapter.py covering all scenarios.\n\nAcceptance Criteria:\n- Test enabled mode (server available)\n- Test disabled mode (server unavailable)\n- Test graceful degradation (server dies mid-operation)\n- Test reservation conflicts\n- Test message sending/receiving\n- Mock HTTP server for testing\n- 90%+ code coverage\n\nFile: lib/test_beads_mail_adapter.py","notes":"Test suite completed with 29 comprehensive tests covering:\n- Enabled mode (server available): 10 tests\n- Disabled mode (server unavailable): 2 tests \n- Graceful degradation: 4 tests\n- Reservation conflicts: 2 tests\n- Configuration: 5 tests\n- Health check scenarios: 3 tests\n- HTTP error handling: 3 tests\n\n**Performance**: All tests run in 10ms (fast!)\n\n**Coverage highlights**:\n✅ Server health checks (ok, degraded, error, timeout)\n✅ All API operations (reserve, release, notify, check_inbox, get_reservations)\n✅ HTTP errors (404, 409 conflict, 500, 503)\n✅ Network errors (timeout, connection refused)\n✅ Malformed responses (bad JSON, empty body, plain text errors)\n✅ Environment variable configuration\n✅ Graceful degradation when server dies mid-operation\n✅ Conflict handling with both JSON and plain text errors\n✅ Dict wrapper responses ({\"messages\": [...]} and {\"reservations\": [...]})\n✅ Custom TTL for reservations\n✅ Default agent name fallback\n\nNo external dependencies, no slow integration tests, just fast unit tests with mocks.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-11-07T22:43:21.294596-08:00","updated_at":"2025-11-08T01:52:23.359176-08:00","closed_at":"2025-11-08T01:32:39.906342-08:00","source_repo":".","dependencies":[{"issue_id":"bd-5ki8","depends_on_id":"bd-m9th","type":"blocks","created_at":"2025-11-07T22:43:21.296024-08:00","created_by":"daemon"}]} {"id":"bd-5ohb","content_hash":"c7432e58e6518764c9e88b92b16becbb545e179f187becf13aa980f1b93c8192","title":"Issue to reopen with reason","description":"","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-11-07T19:00:18.583903-08:00","updated_at":"2025-11-08T02:06:17.805253-08:00","closed_at":"2025-11-08T02:06:17.805253-08:00","source_repo":"."} {"id":"bd-5ots","content_hash":"ba3efab3e7a2b9bb2bd2dba3aace56cfbdd1b67efd1cfc4758d9c79919f632af","title":"SearchIssues N+1 query causes context timeout with GetLabels","description":"scanIssues() calls GetLabels in a loop for every issue, causing N+1 queries and context deadline exceeded errors when used with short timeouts or in-memory databases. This is especially problematic since SearchIssues already supports label filtering via SQL WHERE clauses.","acceptance_criteria":"- Optimize scanIssues to batch-load labels for all issues in one query\n- Or make label loading optional/lazy\n- Add test that calls SearchIssues repeatedly with label filters and short context timeouts","status":"closed","priority":1,"issue_type":"bug","created_at":"2025-11-05T19:12:02.245879-08:00","updated_at":"2025-11-05T19:22:11.668682-08:00","closed_at":"2025-11-05T19:22:11.668682-08:00","source_repo":"."} -{"id":"bd-5qim","content_hash":"5117e87c5a56b5b8254402d982e85bea1478c1961f05654a50cf2df11e7ad6bf","title":"Optimize GetReadyWork performance - 752ms on 10K database (target: \u003c50ms)","description":"","notes":"# Performance Analysis (10K Issue Database)\n\nAnalyzed using CPU profiles from benchmark suite on Apple M2 Pro.\n\n## Operation Performance\n\n| Operation | Time | Allocations | Memory |\n|----------------------------------|---------|-------------|--------|\n| bd ready (GetReadyWork) | ~752ms | 167,466 | 16MB |\n| bd list (SearchIssues no filter) | ~11.6ms | 89,214 | 5.8MB |\n| bd list (SearchIssues filtered) | ~9.2ms | 62,365 | 3.5MB |\n| bd create (CreateIssue) | ~2.6ms | 146 | 8.6KB |\n| bd update (UpdateIssue) | ~0.32ms | 364 | 15KB |\n| bd close (UpdateIssue) | ~0.32ms | 364 | 15KB |\n\n**Target: \u003c50ms for all operations on 10K database**\n\n**Current issue: GetReadyWork is 15x over target (752ms vs 50ms)**\n\n## Root Cause\n\nGetReadyWork (internal/storage/sqlite/ready.go:90-128) uses recursive CTE to propagate blocking:\n- 65x slower than SearchIssues\n- Recalculates entire blocked issue tree on every call\n- Algorithm:\n 1. Find directly blocked issues via 'blocks' dependencies\n 2. Recursively propagate blockage to descendants (max depth: 50)\n 3. Exclude all blocked issues from results\n\n## CPU Profile Analysis\n\n- Database syscalls (pthread_cond_signal, syscall6): ~75%\n- SQLite engine overhead: inherent to recursive CTE\n- Application code (query construction): \u003c1%\n\n**Bottleneck is the recursive CTE query execution, not application code.**\n\n## Optimization Recommendations\n\n### High Impact (Likely to achieve \u003c50ms target)\n\n1. **Cache blocked issue calculation**\n - Add `blocked_issues` table updated on dependency changes\n - Trade write complexity for read speed (ready called \u003e\u003e dependency changes)\n - Eliminates recursive CTE on every read\n\n2. **Add/verify database indexes**\n ```sql\n CREATE INDEX IF NOT EXISTS idx_dependencies_blocked \n ON dependencies(issue_id, type, depends_on_id);\n CREATE INDEX IF NOT EXISTS idx_issues_status \n ON issues(status);\n ```\n\n### Medium Impact\n\n3. **Reduce allocations** (167K allocations for GetReadyWork)\n - Profile `scanIssues()` for object pooling opportunities\n - Reuse slice capacity for repeated calls\n\n### Low Impact (Not recommended)\n- Query optimization for CRUD operations (already \u003c3ms)\n- Connection pooling tuning (not showing in profiles)\n\n## Verification\n\nRun benchmarks to validate optimization:\n```bash\nmake bench-quick\ngo tool pprof -http=:8080 internal/storage/sqlite/bench-cpu-*.prof\n```\n\nProfile files automatically generated in `internal/storage/sqlite/`.","status":"open","priority":0,"issue_type":"bug","created_at":"2025-11-14T09:02:46.507526-08:00","updated_at":"2025-11-14T09:03:44.073236-08:00","source_repo":"."} +{"id":"bd-5qim","content_hash":"5117e87c5a56b5b8254402d982e85bea1478c1961f05654a50cf2df11e7ad6bf","title":"Optimize GetReadyWork performance - 752ms on 10K database (target: \u003c50ms)","description":"","notes":"# Performance Analysis (10K Issue Database)\n\nAnalyzed using CPU profiles from benchmark suite on Apple M2 Pro.\n\n## Operation Performance\n\n| Operation | Time | Allocations | Memory |\n|----------------------------------|---------|-------------|--------|\n| bd ready (GetReadyWork) | ~752ms | 167,466 | 16MB |\n| bd list (SearchIssues no filter) | ~11.6ms | 89,214 | 5.8MB |\n| bd list (SearchIssues filtered) | ~9.2ms | 62,365 | 3.5MB |\n| bd create (CreateIssue) | ~2.6ms | 146 | 8.6KB |\n| bd update (UpdateIssue) | ~0.32ms | 364 | 15KB |\n| bd close (UpdateIssue) | ~0.32ms | 364 | 15KB |\n\n**Target: \u003c50ms for all operations on 10K database**\n\n**Current issue: GetReadyWork is 15x over target (752ms vs 50ms)**\n\n## Root Cause\n\nGetReadyWork (internal/storage/sqlite/ready.go:90-128) uses recursive CTE to propagate blocking:\n- 65x slower than SearchIssues\n- Recalculates entire blocked issue tree on every call\n- Algorithm:\n 1. Find directly blocked issues via 'blocks' dependencies\n 2. Recursively propagate blockage to descendants (max depth: 50)\n 3. Exclude all blocked issues from results\n\n## CPU Profile Analysis\n\n- Database syscalls (pthread_cond_signal, syscall6): ~75%\n- SQLite engine overhead: inherent to recursive CTE\n- Application code (query construction): \u003c1%\n\n**Bottleneck is the recursive CTE query execution, not application code.**\n\n## Optimization Recommendations\n\n### High Impact (Likely to achieve \u003c50ms target)\n\n1. **Cache blocked issue calculation**\n - Add `blocked_issues` table updated on dependency changes\n - Trade write complexity for read speed (ready called \u003e\u003e dependency changes)\n - Eliminates recursive CTE on every read\n\n2. **Add/verify database indexes**\n ```sql\n CREATE INDEX IF NOT EXISTS idx_dependencies_blocked \n ON dependencies(issue_id, type, depends_on_id);\n CREATE INDEX IF NOT EXISTS idx_issues_status \n ON issues(status);\n ```\n\n### Medium Impact\n\n3. **Reduce allocations** (167K allocations for GetReadyWork)\n - Profile `scanIssues()` for object pooling opportunities\n - Reuse slice capacity for repeated calls\n\n### Low Impact (Not recommended)\n- Query optimization for CRUD operations (already \u003c3ms)\n- Connection pooling tuning (not showing in profiles)\n\n## Verification\n\nRun benchmarks to validate optimization:\n```bash\nmake bench-quick\ngo tool pprof -http=:8080 internal/storage/sqlite/bench-cpu-*.prof\n```\n\nProfile files automatically generated in `internal/storage/sqlite/`.","status":"closed","priority":0,"issue_type":"bug","created_at":"2025-11-14T09:02:46.507526-08:00","updated_at":"2025-11-20T19:26:38.69035-05:00","closed_at":"2025-11-20T19:26:38.69035-05:00","source_repo":"."} {"id":"bd-6","content_hash":"d2873aaf1e25e6b17fb4e9caeb6b31e42e2bc2783e98a68d4096b2f6f62fb90e","title":"Add --show-all-paths flag to bd dep tree","description":"Currently bd dep tree deduplicates nodes when multiple paths exist (diamond dependencies). Add optional --show-all-paths flag to display the full graph with all paths, showing duplicates. Useful for debugging complex dependency structures and understanding all relationships.","status":"closed","priority":3,"issue_type":"feature","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-21T00:25:40.544198-07:00","closed_at":"2025-10-18T10:11:38.985862-07:00","source_repo":"."} {"id":"bd-60","content_hash":"317b9062c98a7d890d5c5fc168427dcc44d98006559b57ece5fdcfbb67cf7491","title":"Implement reference scoring algorithm","description":"Count references for each colliding issue: text mentions in descriptions/notes/design fields + dependency references. Sort collisions by score ascending (fewest refs first). This minimizes total updates during renumbering.","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-16T20:46:08.971822-07:00","updated_at":"2025-10-21T00:25:40.544351-07:00","closed_at":"2025-10-14T02:51:52.198288-07:00","source_repo":"."} {"id":"bd-600","content_hash":"a5d54da18f360fddf7b13701da4e7687ce7497044265ec7a478aed7d23d6c02e","title":"Agent 2 Issue 1","description":"Created by agent 2","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-21T00:29:10.623632-07:00","updated_at":"2025-10-21T00:29:10.623632-07:00","source_repo":"."}