Commit Graph

207 Commits

Author SHA1 Message Date
Steve Yegge
331a435418 Fix collision resolver: sync ID counters before remapping
The collision resolver was failing when remapping issues to new IDs because
the issue_counters table was out of sync with actual database IDs. Test issues
(bd-200 through bd-312) were created without updating the counter, causing
UNIQUE constraint violations when --resolve-collisions tried to use those IDs.

Added SyncAllCounters() call before remapping to ensure counters reflect all
existing issues in the database. This prevents ID collisions during import.

Fixes the core functionality needed for multi-device git-based sync (bd-271).

Amp-Thread-ID: https://ampcode.com/threads/T-a2a94e1b-b220-41b0-bf7d-f2640a44292b
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 11:59:34 -07:00
Steve Yegge
80a53833e1 Close bd-84: SHA256 hash-based auto-import implemented and tested 2025-10-16 11:09:56 -07:00
Steve Yegge
b7f9e74574 feat: Implement cross-type cycle prevention in dependencies
- Remove type filter from cycle detection to check ALL dependency types
- Extract maxDependencyDepth=100 constant shared across AddDependency and DetectCycles
- Move cycle check before INSERT to avoid unnecessary write on failure
- Add comprehensive tests: self-dependency, related cycles, cross-type cycles
- Verify idx_dependencies_issue index exists for performance

Fixes bd-312. Prevents cross-type cycles (e.g., A blocks B, B parent-child A)
that previously hid work from ready list. Addresses oracle feedback for
proper implementation.
2025-10-16 10:59:25 -07:00
Steve Yegge
8541203178 added epic for git jsonl auto-syncing 2025-10-16 01:41:23 -07:00
Steve Yegge
1cc37d9acf Remove all restore/snapshot references from compaction
- Removed restore command from bd show output
- Updated compact help text (removed snapshot claim)
- Fixed COMPACTION.md (removed 'batch restore' from roadmap)
- All compaction UI now correctly states permanent decay
2025-10-16 01:13:08 -07:00
Steve Yegge
ebcc503849 Fix compaction docs: remove non-existent restore feature
Compaction is permanent graceful decay - restore functionality
doesn't exist. Updated docs to reflect reality:
- Removed restore command examples
- Removed snapshot/restore sections
- Updated to explain git history recovery option
- Clarified permanent nature throughout
2025-10-16 01:11:35 -07:00
Steve Yegge
54469936a7 Add comprehensive compaction documentation
- Updated README.md with Tier 1/2 info, restore command, cost analysis
- Created COMPACTION.md with full guide covering:
  - How compaction works (architecture, two-tier system)
  - CLI reference and examples
  - Eligibility rules and configuration
  - Cost analysis with detailed tables
  - Automation examples (cron, workflows)
  - Safety, recovery, and troubleshooting
  - FAQ and best practices
- Added examples/compaction/ with 3 scripts:
  - workflow.sh: Interactive compaction workflow
  - cron-compact.sh: Automated monthly compaction
  - auto-compact.sh: Smart threshold-based compaction
  - README.md: Examples documentation

Closes bd-265

Amp-Thread-ID: https://ampcode.com/threads/T-8113e88e-1cd0-4a9e-b581-07045a3ed31e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 01:09:48 -07:00
Steve Yegge
6786d68365 Add compaction status indicators to bd show command
- Display emoji in title line (🗜️ Tier 1, 📦 Tier 2)
- Show compaction level in status line
- Add footer with size savings and restore command
- JSON output includes all compaction fields

Closes bd-263

Amp-Thread-ID: https://ampcode.com/threads/T-8113e88e-1cd0-4a9e-b581-07045a3ed31e
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 01:04:59 -07:00
Steve Yegge
e99de1b5e3 Add EventCompacted to event system (bd-262)
- Add EventCompacted event type constant
- Add compaction fields to Issue struct (CompactionLevel, CompactedAt, OriginalSize)
- Update ApplyCompaction to record compaction events with JSON metadata
- Update bd show to display compaction status with emoji indicators
- Update GetIssue query to load compaction fields
- All tests passing

Amp-Thread-ID: https://ampcode.com/threads/T-3f7946c6-8f5e-4a81-9527-1217041c7b39
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 01:00:27 -07:00
Steve Yegge
5e420e8ee0 Fix bd-270: Detect and handle Git merge conflicts in JSONL auto-import
- Add pre-parse merge conflict marker detection before JSON parsing
- Show clear error message when conflicts are detected
- Provide resolution instructions (Git client or bd export)
- Add TestAutoImportMergeConflict test case
- Prevents silent auto-import failures from cryptic parse errors

Amp-Thread-ID: https://ampcode.com/threads/T-d83011c0-7dfc-49c9-96c1-05c94ec2a3d3
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 00:49:41 -07:00
Steve Yegge
392379efad Document compaction feature in README
Added light documentation for bd compact command:
- Feature list entry: Memory decay
- Usage section with examples
- Requirements and eligibility criteria
- Emphasizes intentional graceful decay vs reversible compression
2025-10-16 00:29:10 -07:00
Steve Yegge
da5493bac0 Remove snapshot/restore functionality from compaction
Snapshots defeated the entire purpose of compaction - if we're keeping
the original content, we're not actually saving any space. Compaction
is about graceful memory decay for agentic databases, not reversible
compression.

Removed:
- CreateSnapshot/GetSnapshots/RestoreFromSnapshot from storage
- --restore flag and functionality from bd compact command
- All snapshot-related tests
- Snapshot struct and related code

The database is ephemeral and meant to decay over time. Compaction
actually reduces database size now.

Closes bd-260 (won't fix - conceptually wrong)
Closes bd-261 (already done in bd-259)
2025-10-16 00:26:42 -07:00
Steve Yegge
7fe6bf3e1d Close bd-268: Investigation complete, stick with modernc.org/sqlite
Amp-Thread-ID: https://ampcode.com/threads/T-820bd234-e2ca-4aca-a516-b46aae92c222
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 00:25:29 -07:00
Steve Yegge
35a4cba829 Add bd compact CLI command (bd-259)
Implements the compact command with all required features:
- --dry-run: Preview compaction with size estimates
- --all: Process all eligible candidates
- --id: Compact specific issue
- --force: Bypass eligibility checks (requires --id)
- --stats: Show compaction statistics
- --tier: Select compaction tier (1 or 2)
- --workers: Configure parallel workers
- --batch-size: Configure batch processing
- Progress bar with visual feedback
- JSON output support
- Proper exit codes and error handling
- Summary reporting (count, bytes saved, reduction %, time)

Includes additional test coverage for compaction and snapshot operations.

Amp-Thread-ID: https://ampcode.com/threads/T-ffcaf749-f79c-4b03-91dd-42136b2744b1
Co-authored-by: Amp <amp@ampcode.com>
2025-10-16 00:13:14 -07:00
Steve Yegge
eb47f4f26c Update issues: close bd-269, add worktree/cli labels 2025-10-15 23:53:12 -07:00
Steve Yegge
e2c42c35c4 Add label and title filtering to bd list (closes #45, bd-269)
- Add --label/-l flag to filter issues by labels (AND logic)
- Add --title flag to filter issues by title substring
- Add TitleSearch field to IssueFilter type
- Implement label and title filtering in SearchIssues
- Perfect for worktree-specific issue management

Examples:
  bd list --label worktree,feature-x
  bd list --title "authentication"
  bd list --label worktree --title "bug"
2025-10-15 23:51:57 -07:00
Steve Yegge
30c774a976 Merge PR #46: Add --format flag to bd list
Adds flexible formatting to bd list command:
- digraph format for golang.org/x/tools/cmd/digraph
- dot format for Graphviz visualization
- Custom Go templates for output

Co-authored-by: Travis Cline <travis.cline@gmail.com>
Amp-Thread-ID: https://ampcode.com/threads/T-7f272fc9-3778-48cf-99b5-9b1f0cbbc5dc
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 23:45:00 -07:00
Steve Yegge
cdd24699be Merge refactoring PR #48: Reduce cyclomatic complexity
Co-authored-by: jjshanks <jjshanks@users.noreply.github.com>
2025-10-15 23:39:24 -07:00
Joshua Shanks
96f05a79cb Fix linter errors in CreateIssues after upstream merge
- Fix unchecked error return in CreateIssues ROLLBACK (errcheck)
- Reduce cyclomatic complexity of CreateIssues from 28 to ~10 (gocyclo)
  - Extract validateBatchIssues helper for validation phase
  - Extract generateBatchIDs helper for ID generation
  - Extract bulkInsertIssues helper for issue insertion
  - Extract bulkRecordEvents helper for event recording
  - Extract bulkMarkDirty helper for dirty marking

All refactored helpers maintain the same functionality and transaction safety.
Follows the same pattern used in main.go complexity reduction.
2025-10-15 23:38:47 -07:00
Joshua Shanks
f2e91f1858 Trigger CI re-run - linter errors already fixed in a517601 2025-10-15 23:38:47 -07:00
Joshua Shanks
b1e8ef556e Refactor main.go to reduce cyclomatic complexity
Breaks down large functions into smaller, focused helpers to pass gocyclo linter:

Auto-import refactoring:
- Extract parseJSONLIssues() to handle JSONL parsing
- Extract handleCollisions() to detect and report conflicts
- Extract importIssueData() to coordinate issue/dep/label imports
- Extract updateExistingIssue() and createNewIssue() for clarity
- Extract importDependencies() and importLabels() for modularity

Flush refactoring:
- Extract recordFlushFailure() and recordFlushSuccess() for state management
- Extract readExistingJSONL() to isolate file reading logic
- Extract fetchDirtyIssuesFromDB() to separate DB access
- Extract writeIssuesToJSONL() to handle atomic writes

Command improvements:
- Extract executeLabelCommand() to eliminate duplication in label.go
- Extract addLabelsToIssue() helper for label management
- Replace deprecated strings.Title with manual capitalization

Configuration:
- Add gocyclo exception for test files in .golangci.yml

All tests passing, no functionality changes.
2025-10-15 23:38:47 -07:00
Joshua Shanks
cf4f11cff7 Fix error handling consistency in auto-import and fallback paths (#47)
* Fix error handling consistency in auto-import and fallback paths

- Add error checking/warnings for auto-import CRUD operations (UpdateIssue, CreateIssue, AddDependency)
- Add error checking/warnings for auto-import label operations (AddLabel, RemoveLabel)
- Add warning when import hash storage fails (prevents unnecessary re-imports)
- Add proper error handling for UserHomeDir with fallback to current directory

These changes make auto-import error handling consistent with manual operations
and prevent silent failures that could confuse users or cause data inconsistencies.

* Remove invalid version property from golangci-lint config

* Fix linter errors: errcheck, unused, goconst, and misspell

- Fix unchecked error returns in ROLLBACK statements
- Fix unchecked type assertion for status field
- Extract LIMIT SQL constant to reduce duplication
- Fix spelling: cancelled -> canceled
- Remove unused ensureCounterInitialized function
- Remove unused parameter in parallel test goroutine
2025-10-15 23:34:33 -07:00
Steve Yegge
0da81371b4 Implement Tier 1 compaction logic (bd-257)
- Add Compactor with CompactTier1 and CompactTier1Batch methods
- Single issue and batch compaction with 5 concurrent workers
- Dry-run mode for testing without API calls
- Smart size checking: keeps original if summary is longer
- Improved Haiku prompts to emphasize compression
- Add ApplyCompaction method for setting compaction metadata
- Comprehensive tests including API integration tests
- All tests passing
2025-10-15 23:31:43 -07:00
Steve Yegge
5f6aac5fb1 Implement snapshot creation and restoration for compaction (bd-256)
- Add compaction_snapshots table to schema with proper indexes
- Implement CreateSnapshot, RestoreFromSnapshot, GetSnapshots functions
- Use UTC timestamps throughout
- RestoreFromSnapshot uses transactions with optimistic concurrency control
- Add validation for levels and issue_id matching
- Prevent race conditions with compaction_level guard
- Create bd-268 to explore lightweight SQL alternatives

Amp-Thread-ID: https://ampcode.com/threads/T-3bdd0d6b-9212-4e4e-b22d-f658949df7a9
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 23:20:21 -07:00
Steve Yegge
3e3f46d6d2 Add Claude Haiku client for issue compaction (bd-255)
- Implement HaikuClient with Tier 1 and Tier 2 summarization prompts
- Add exponential backoff retry logic for transient errors (network, 429, 5xx)
- API key precedence: env var overrides explicit key parameter
- Comprehensive tests with UTF-8 support and edge case coverage
- Retry logic only retries transient errors, stops on permanent failures
- Clarified retry semantics: maxRetries=3 means 4 total attempts (1 initial + 3 retries)
2025-10-15 23:09:33 -07:00
Steve Yegge
1c5a4a9c70 Add compaction schema and candidate identification
- Added compaction columns to issues table (compaction_level, compacted_at, original_size)
- Created issue_snapshots table for snapshot storage before compaction
- Added compaction configuration with opt-in flag (compaction_enabled=false by default)
- Implemented GetTier1Candidates and GetTier2Candidates queries
- Added CheckEligibility validation function
- Comprehensive tests for all compaction queries
- Idempotent migrations for existing databases

Closes bd-252, bd-253, bd-254

Amp-Thread-ID: https://ampcode.com/threads/T-c4d7acd1-c161-4b80-9d80-a0691e8fa87b
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 22:26:11 -07:00
Steve Yegge
eaf08106c1 Remove compaction planning docs (issues filed as bd-251 to bd-267) 2025-10-15 21:54:03 -07:00
Steve Yegge
0817b44469 Add compaction feature design and file 17 issues (bd-251 to bd-267)
Amp-Thread-ID: https://ampcode.com/threads/T-8535178e-f814-43e7-a8a0-4aea93ef3970
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 21:53:08 -07:00
Steve Yegge
61d70f208f removed obsolete docs 2025-10-15 21:16:52 -07:00
Steve Yegge
dfc62054f1 issue updates 2025-10-15 21:15:51 -07:00
Steve Yegge
512996b87a Update issues: close bd-243, bd-245, bd-247 2025-10-15 20:24:16 -07:00
Steve Yegge
347100319e Document CreateIssues API (bd-243)
- Add comprehensive godoc to CreateIssues with usage examples
- Add batch operations section to EXTENDING.md with performance comparison
- Add performance feature to README.md
- Include when to use CreateIssue vs CreateIssues guidance
- Document counter sync requirement after explicit IDs

Amp-Thread-ID: https://ampcode.com/threads/T-59fc78c3-a7f2-4c33-b074-2fa840c97c87
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 20:24:07 -07:00
Steve Yegge
58bdf4bc0b Update issues: close bd-222 (CreateIssues complete) 2025-10-15 20:20:20 -07:00
Steve Yegge
de236d0d1f Fix skip-existing logic for batch import duplicates
The skipUpdate flag should only apply to existing DB issues, not to
duplicates within the import batch. Batch duplicates always use
last-one-wins semantics.
2025-10-15 20:09:24 -07:00
Steve Yegge
3eb4f4d040 Optimize import with batch CreateIssues (bd-242)
- Replace CreateIssue loop with single CreateIssues batch call
- Add in-memory de-duplication for duplicate IDs (last one wins)
- 5-15x faster for bulk imports
- All tests pass
2025-10-15 19:57:46 -07:00
Steve Yegge
0a81b08c4f chore: Close bd-71 code review follow-up epic
Amp-Thread-ID: https://ampcode.com/threads/T-8a764023-ed80-4333-a037-46936e5e5d08
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 19:41:57 -07:00
Steve Yegge
5ab7ff0f84 Add comprehensive unit tests for CreateIssues (bd-241)
- Added 12 table-driven test cases covering all validation scenarios
- Added 2 rollback tests (validation error + DB conflict)
- Added nil item detection to prevent panic
- Fixed nil pointer panic in CreateIssues
- Tests verify ID uniqueness, timestamps, closed_at invariant
- All tests pass

Amp-Thread-ID: https://ampcode.com/threads/T-61c584cd-d873-4a1a-bfa6-d739b630b3e5
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 19:33:53 -07:00
Steve Yegge
e7d4a9c822 Implement CreateIssues batch API (bd-240, bd-244)
- Add CreateIssues method to Storage interface
- Implement SQLiteStorage.CreateIssues with atomic ID range reservation
- Single transaction for N issues (5-10x speedup expected)
- Set timestamps before validation to match CreateIssue behavior
- All tests passing
2025-10-15 19:13:27 -07:00
Steve Yegge
e4394df604 Close bd-224: status/closed_at invariant fully enforced 2025-10-15 18:03:58 -07:00
Steve Yegge
737f354066 Update issue tracking after label test implementation 2025-10-15 17:51:14 -07:00
Steve Yegge
a00c0e2199 Add comprehensive tests for label management commands
- Test add/remove labels
- Test multiple labels and duplicates
- Test label persistence across updates
- Test event creation for label operations
- Test labels with different issue types
- All 10 test cases passing
2025-10-15 17:51:09 -07:00
Steve Yegge
3c8e4d78e5 Add label management and import collision detection
- Implement 'bd label' command with add/remove/list subcommands
- Add import --dry-run and --resolve-collisions for safe merges
- Support label filtering in 'bd list' and 'bd create'
- Update AGENTS.md with collision handling workflow
- Extends bd-224 (data consistency) and bd-84 (import reliability)
2025-10-15 17:19:16 -07:00
Steve Yegge
6bfaad8731 Add reopen command tests (bd-248)
Amp-Thread-ID: https://ampcode.com/threads/T-7c3a7351-f785-440d-8200-da858cf98426
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 17:06:59 -07:00
Steve Yegge
f78f529087 Add bd reopen command for explicit issue reopening
Implements bd-235: New reopen command makes reopening issues more explicit
than 'bd update --status open'.

Features:
- Sets status to 'open' and automatically clears closed_at
- Emits EventReopened via UpdateIssue
- Optional --reason flag to add comment
- JSON output support
- Auto-flush after operation

Closes: bd-234, bd-235, bd-236, bd-237, bd-238, bd-239
Related: bd-224 (status/closed_at invariant enforcement)
Amp-Thread-ID: https://ampcode.com/threads/T-b1c82629-3521-440f-9f16-ab6d66fb5091
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 16:31:11 -07:00
Travis Cline
4131e6bf38 list: add status-based color coding to dot format
Enhance Graphviz dot output with status-based fill colors:
- open: white background (default)
- in_progress: light yellow background
- blocked: light coral background
- closed: light gray background with dimmed text

Node labels show: ID, type, priority, title, and status.
Priority is visible in the label (e.g., [bug P0]) but not color-coded
to keep the visualization clean and focused on status.
2025-10-15 15:27:49 -07:00
Travis Cline
6f357ea536 list: add --format flag with template support
Add flexible --format flag to 'bd list' command supporting:
- Built-in presets: 'digraph' (basic 'from to' format) and 'dot' (Graphviz)
- Custom Go templates for dependency output
- Template variables: IssueID, DependsOnID, Type, Issue, Dependency

This enables graph analysis with tools like golang.org/x/tools/cmd/digraph while allowing users to customize output format for their specific needs.

Examples:
  bd list --format=digraph | digraph nodes
  bd list --format=dot | dot -Tsvg -o deps.svg
  bd list --format='{{.IssueID}} -> {{.DependsOnID}} [{{.Type}}]'
2025-10-15 15:27:24 -07:00
Travis Cline
0c3c613890 list: add filter value enums to help 2025-10-15 15:27:24 -07:00
Travis Cline
cf8ff4227d list: extract command to separate file 2025-10-15 15:27:24 -07:00
Steve Yegge
d2b50e6cdc Add closed_at timestamp tracking to issues
- Add closed_at field to Issue type with JSON marshaling
- Implement closed_at timestamp in SQLite storage layer
- Update import/export to handle closed_at field
- Add comprehensive tests for closed_at functionality
- Maintain backward compatibility with existing databases

Amp-Thread-ID: https://ampcode.com/threads/T-f3a7799b-f91e-4432-a690-aae0aed364b3
Co-authored-by: Amp <amp@ampcode.com>
2025-10-15 14:52:29 -07:00
Steve Yegge
ab809c5baf Create issue structure for bd-222 and bd-224
- Added design documents (ULTRATHINK_BD222.md, ULTRATHINK_BD224.md)
- Created bd-224 epic with 6 child issues for status/closed_at invariant fix
- Created bd-222 epic with 7 child issues for batching API
- Set up dependencies: bd-224 blocks bd-222 (must fix invariant first)
- Dependencies enable max parallelism while ensuring correct order
2025-10-15 14:27:10 -07:00