johno/beads
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Claude Haiku client for issue compaction (bd-255)

Browse Source 
- Implement HaikuClient with Tier 1 and Tier 2 summarization prompts
- Add exponential backoff retry logic for transient errors (network, 429, 5xx)
- API key precedence: env var overrides explicit key parameter
- Comprehensive tests with UTF-8 support and edge case coverage
- Retry logic only retries transient errors, stops on permanent failures
- Clarified retry semantics: maxRetries=3 means 4 total attempts (1 initial + 3 retries)
This commit is contained in:
Steve Yegge
2025-10-15 23:09:33 -07:00
parent 1c5a4a9c70
commit 3e3f46d6d2
5 changed files with 492 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.beads/issues.jsonl
View File

@@ -170,7 +170,7 @@
{"id":"bd-252","title":"Add compaction schema and migrations","description":"Add database schema support for issue compaction tracking and snapshot storage.","design":"Add three columns to `issues` table:\n- `compaction_level INTEGER DEFAULT 0` - 0=original, 1=tier1, 2=tier2\n- `compacted_at DATETIME` - when last compacted\n- `original_size INTEGER` - bytes before first compaction\n\nCreate `issue_snapshots` table:\n```sql\nCREATE TABLE issue_snapshots (\n id INTEGER PRIMARY KEY AUTOINCREMENT,\n issue_id TEXT NOT NULL,\n snapshot_time DATETIME NOT NULL,\n compaction_level INTEGER NOT NULL,\n original_size INTEGER NOT NULL,\n compressed_size INTEGER NOT NULL,\n original_content TEXT NOT NULL, -- JSON blob\n archived_events TEXT,\n FOREIGN KEY (issue_id) REFERENCES issues(id) ON DELETE CASCADE\n);\n```\n\nAdd indexes:\n- `idx_snapshots_issue` on `issue_id`\n- `idx_snapshots_level` on `compaction_level`\n\nAdd migration functions in `internal/storage/sqlite/sqlite.go`:\n- `migrateCompactionColumns(db *sql.DB) error`\n- `migrateSnapshotsTable(db *sql.DB) error`","acceptance_criteria":"- Existing databases migrate automatically\n- New databases include columns by default\n- Migration is idempotent (safe to run multiple times)\n- No data loss during migration\n- Tests verify migration on fresh and existing DBs","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.216371-07:00","updated_at":"2025-10-15T22:02:27.638283-07:00","closed_at":"2025-10-15T22:02:27.638283-07:00"}
{"id":"bd-253","title":"Add compaction configuration keys","description":"Add configuration keys for compaction behavior with sensible defaults.","design":"Add to `internal/storage/sqlite/schema.go` initial config:\n```sql\nINSERT OR IGNORE INTO config (key, value) VALUES\n ('compact_tier1_days', '30'),\n ('compact_tier1_dep_levels', '2'),\n ('compact_tier2_days', '90'),\n ('compact_tier2_dep_levels', '5'),\n ('compact_tier2_commits', '100'),\n ('compact_model', 'claude-3-5-haiku-20241022'),\n ('compact_batch_size', '50'),\n ('compact_parallel_workers', '5'),\n ('auto_compact_enabled', 'false');\n```\n\nAdd helper functions for loading config into typed struct.","acceptance_criteria":"- Config keys created on init\n- Existing DBs get defaults on migration\n- `bd config get/set` works with all keys\n- Type validation (days=int, enabled=bool)\n- Documentation in README.md","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.22391-07:00","updated_at":"2025-10-15T22:08:44.984927-07:00","closed_at":"2025-10-15T22:08:44.984927-07:00"}
{"id":"bd-254","title":"Implement candidate identification queries","description":"Write SQL queries to identify issues eligible for Tier 1 and Tier 2 compaction based on closure time and dependency status.","design":"Create `internal/storage/sqlite/compact.go` with:\n\n```go\ntype CompactionCandidate struct {\n IssueID string\n ClosedAt time.Time\n OriginalSize int\n EstimatedSize int\n DependentCount int\n}\n\nfunc (s *SQLiteStorage) GetTier1Candidates(ctx context.Context) ([]*CompactionCandidate, error)\nfunc (s *SQLiteStorage) GetTier2Candidates(ctx context.Context) ([]*CompactionCandidate, error)\nfunc (s *SQLiteStorage) CheckEligibility(ctx context.Context, issueID string, tier int) (bool, string, error)\n```\n\nUse recursive CTE for dependency depth checking (similar to ready_issues view).","acceptance_criteria":"- Tier 1 query filters by days and dependency depth\n- Tier 2 query includes commit/issue count checks\n- Dependency checking handles circular deps gracefully\n- Performance: \u003c100ms for 10,000 issue database\n- Tests cover edge cases (no deps, circular deps, mixed status)","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.225835-07:00","updated_at":"2025-10-15T22:16:45.517562-07:00","closed_at":"2025-10-15T22:16:45.517562-07:00"}
{"id":"bd-255","title":"Create Haiku client and prompt templates","description":"Implement Claude Haiku API client with template-based prompts for Tier 1 and Tier 2 summarization.","design":"Create `internal/compact/haiku.go`:\n\n```go\ntype HaikuClient struct {\n client *anthropic.Client\n model string\n}\n\nfunc NewHaikuClient(apiKey string) (*HaikuClient, error)\nfunc (h *HaikuClient) SummarizeTier1(ctx context.Context, issue *types.Issue) (string, error)\nfunc (h *HaikuClient) SummarizeTier2(ctx context.Context, issue *types.Issue) (string, error)\n```\n\nUse text/template for prompt rendering.\n\nTier 1 output format:\n```\n**Summary:** [2-3 sentences]\n**Key Decisions:** [bullet points]\n**Resolution:** [outcome]\n```\n\nTier 2 output format:\n```\nSingle paragraph ≤150 words covering what was built, why it mattered, lasting impact.\n```","acceptance_criteria":"- API key from env var or config (env takes precedence)\n- Prompts render correctly with templates\n- Rate limiting handled gracefully (exponential backoff)\n- Network errors retry up to 3 times\n- Mock tests for API calls","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.229702-07:00","updated_at":"2025-10-15T21:51:23.229702-07:00"}
{"id":"bd-255","title":"Create Haiku client and prompt templates","description":"Implement Claude Haiku API client with template-based prompts for Tier 1 and Tier 2 summarization.","design":"Create `internal/compact/haiku.go`:\n\n```go\ntype HaikuClient struct {\n client *anthropic.Client\n model string\n}\n\nfunc NewHaikuClient(apiKey string) (*HaikuClient, error)\nfunc (h *HaikuClient) SummarizeTier1(ctx context.Context, issue *types.Issue) (string, error)\nfunc (h *HaikuClient) SummarizeTier2(ctx context.Context, issue *types.Issue) (string, error)\n```\n\nUse text/template for prompt rendering.\n\nTier 1 output format:\n```\n**Summary:** [2-3 sentences]\n**Key Decisions:** [bullet points]\n**Resolution:** [outcome]\n```\n\nTier 2 output format:\n```\nSingle paragraph ≤150 words covering what was built, why it mattered, lasting impact.\n```","acceptance_criteria":"- API key from env var or config (env takes precedence)\n- Prompts render correctly with templates\n- Rate limiting handled gracefully (exponential backoff)\n- Network errors retry up to 3 times\n- Mock tests for API calls","status":"closed","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.229702-07:00","updated_at":"2025-10-15T22:32:51.491798-07:00","closed_at":"2025-10-15T22:32:51.491798-07:00"}
{"id":"bd-256","title":"Implement snapshot creation and restoration","description":"Implement snapshot creation before compaction and restoration capability to undo compaction.","design":"Add to `internal/storage/sqlite/compact.go`:\n\n```go\nfunc (s *SQLiteStorage) CreateSnapshot(ctx context.Context, issue *types.Issue, level int) error\nfunc (s *SQLiteStorage) RestoreFromSnapshot(ctx context.Context, issueID string, level int) error\nfunc (s *SQLiteStorage) GetSnapshots(ctx context.Context, issueID string) ([]*Snapshot, error)\n```\n\nSnapshot JSON structure:\n```json\n{\n \"description\": \"...\",\n \"design\": \"...\",\n \"notes\": \"...\",\n \"acceptance_criteria\": \"...\",\n \"title\": \"...\"\n}\n```","acceptance_criteria":"- Snapshot created atomically with compaction\n- Restore returns exact original content\n- Multiple snapshots per issue supported (Tier 1 → Tier 2)\n- JSON encoding handles UTF-8 and special characters\n- Size calculation is accurate (UTF-8 bytes)","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.231906-07:00","updated_at":"2025-10-15T21:51:23.231906-07:00"}
{"id":"bd-257","title":"Implement Tier 1 compaction logic","description":"Implement the core Tier 1 compaction process: snapshot → summarize → update.","design":"Add to `internal/compact/compactor.go`:\n\n```go\ntype Compactor struct {\n store storage.Storage\n haiku *HaikuClient\n config *CompactConfig\n}\n\nfunc New(store storage.Storage, apiKey string, config *CompactConfig) (*Compactor, error)\nfunc (c *Compactor) CompactTier1(ctx context.Context, issueID string) error\nfunc (c *Compactor) CompactTier1Batch(ctx context.Context, issueIDs []string) error\n```\n\nProcess:\n1. Verify eligibility\n2. Calculate original size\n3. Create snapshot\n4. Call Haiku for summary\n5. Update issue (description=summary, clear design/notes/criteria)\n6. Set compaction_level=1, compacted_at=now, original_size\n7. Record EventCompacted\n8. Mark dirty for export","acceptance_criteria":"- Single issue compaction works end-to-end\n- Batch processing with parallel workers (5 concurrent)\n- Errors don't corrupt database (transaction rollback)\n- EventCompacted includes size savings\n- Dry-run mode (identify + size estimate only, no API calls)","status":"open","priority":1,"issue_type":"task","created_at":"2025-10-15T21:51:23.23391-07:00","updated_at":"2025-10-15T21:51:23.23391-07:00"}
{"id":"bd-258","title":"Implement Tier 2 compaction logic","description":"Implement Tier 2 ultra-compression: more aggressive summarization and optional event pruning.","design":"Add to `internal/compact/compactor.go`:\n\n```go\nfunc (c *Compactor) CompactTier2(ctx context.Context, issueID string) error\nfunc (c *Compactor) CompactTier2Batch(ctx context.Context, issueIDs []string) error\n```\n\nProcess:\n1. Verify issue is at compaction_level = 1\n2. Check Tier 2 eligibility (days, deps, commits/issues)\n3. Create Tier 2 snapshot\n4. Call Haiku with ultra-compression prompt\n5. Update issue (description = single paragraph, clear all other fields)\n6. Set compaction_level = 2\n7. Optionally prune events (keep created/closed, archive rest to snapshot)","acceptance_criteria":"- Requires existing Tier 1 compaction\n- Git commit counting works (with fallback to issue counter)\n- Events optionally pruned (config: compact_events_enabled)\n- Archived events stored in snapshot JSON\n- Size reduction 90-95%","status":"open","priority":2,"issue_type":"task","created_at":"2025-10-15T21:51:23.23586-07:00","updated_at":"2025-10-15T21:51:23.23586-07:00"}