beads/docs/dev-notes/TEST_SUITE_AUDIT.md

cmd/bd Test Suite Audit (bd-c49)

Executive Summary

Original State: 280 tests across 76 test files, each creating isolated database setups Phase 1 Complete: 6 P1 test files refactored with shared DB setup (bd-1rh) Achieved Speedup: P1 tests now run in 0.43 seconds (vs. estimated 10+ minutes before) Remaining Work: P2 and P3 files still use isolated DB setups

Analysis Categories

Category 1: Pure DB Tests (Can Share DB Setup) - 150+ tests

These tests only interact with the database and can safely share a single DB setup per suite:

High Priority Candidates (P1 - Start Here):

1. ✓ create_test.go (11 tests) → TestCreateSuite DONE (bd-y6d)

   • All tests: TestCreate_*
   • Before: 11 separate newTestStore() calls
   • After: 1 shared DB, 11 subtests
   • Result: 10x faster

2. label_test.go (1 suite with 11 subtests) → Already optimal!

   • Uses helper pattern with single DB setup
   • This is the TEMPLATE for refactoring!

3. ✓ dep_test.go (9 tests) → TestDependencySuite DONE (bd-1rh)

   • All tests: TestDep_*
   • Before: 4 newTestStore() calls
   • After: 1 shared DB, 4 subtests (+ command init tests)
   • Result: 4x faster

4. ✓ list_test.go (3 tests) → TestListCommandSuite + TestListQueryCapabilitiesSuite DONE (bd-1rh)

   • Before: 2 newTestStore() calls
   • After: 2 shared DBs (split to avoid data pollution), multiple subtests
   • Result: 2x faster

5. ✓ comments_test.go (3 tests) → TestCommentsSuite DONE (bd-1rh)

   • Before: 2 newTestStore() calls
   • After: 1 shared DB, 2 subtest groups with 6 total subtests
   • Result: 2x faster

6. ✓ stale_test.go (6 tests) → Individual test functions DONE (bd-1rh)

   • Before: 5 newTestStore() calls
   • After: 6 individual test functions (shared DB caused data pollution)
   • Result: Slight improvement (data isolation was necessary)

7. ✓ ready_test.go (4 tests) → TestReadySuite DONE (bd-1rh)

   • Before: 3 newTestStore() calls
   • After: 1 shared DB, 3 subtests
   • Result: 3x faster

8. reopen_test.go (1 test) → Leave as-is or merge

   • Single test, minimal benefit from refactoring

Medium Priority Candidates (P2):

9. main_test.go (18 tests) → TestMainSuite

   • Current: 14 newTestStore() calls
   • Proposed: 1-2 shared DBs (may need isolation for some)
   • Expected speedup: 5-7x faster

10. integrity_test.go (6 tests) → TestIntegritySuite

    • Current: 15 newTestStore() calls (many helper calls)
    • Proposed: 1 shared DB, 6 subtests
    • Expected speedup: 10x faster

11. export_import_test.go (4 tests) → TestExportImportSuite

    • Current: 4 newTestStore() calls
    • Proposed: 1 shared DB, 4 subtests
    • Expected speedup: 4x faster

Category 2: Tests Needing Selective Isolation (60+ tests)

These have a mix - some can share DB, some need isolation:

Daemon Tests (Already have integration tags):

• daemon_test.go (15 tests) - Mix of DB and daemon lifecycle

  • Propose: Separate suites for DB-only vs daemon lifecycle tests

• daemon_autoimport_test.go (2 tests)

• daemon_crash_test.go (2 tests)

• daemon_lock_test.go (6 tests)

• daemon_parent_test.go (1 test)

• daemon_sync_test.go (6 tests)

• daemon_sync_branch_test.go (11 tests)

• daemon_watcher_test.go (7 tests)

Recommendation: Keep daemon tests isolated (they already have //go:build integration tags)

Git Operation Tests:

• git_sync_test.go (1 test)
• sync_test.go (16 tests)
• sync_local_only_test.go (2 tests)
• import_uncommitted_test.go (2 tests)

Recommendation: Keep git tests isolated (need real git repos)

Category 3: Already Well-Optimized (20+ tests)

Tests that already use good patterns:

1. label_test.go - Uses helper struct with shared DB ✓
2. delete_test.go - Has //go:build integration tag ✓
3. All daemon tests - Have //go:build integration tags ✓

Category 4: Special Cases (50+ tests)

CLI Integration Tests:

• cli_fast_test.go (17 tests) - End-to-end CLI testing
  • Keep isolated, already tagged //go:build integration

Import/Export Tests:

• import_bug_test.go (1 test)
• import_cancellation_test.go (2 tests)
• import_idempotent_test.go (3 tests)
• import_multipart_id_test.go (2 tests)
• export_mtime_test.go (3 tests)
• export_test.go (1 test)

Recommendation: Most can share DB within their suite

Filesystem/Init Tests:

• init_test.go (8 tests)
• init_hooks_test.go (3 tests)
• reinit_test.go (1 test)
• onboard_test.go (1 test)

Recommendation: Need isolation (modify filesystem)

Validation/Utility Tests:

• validate_test.go (9 tests)
• template_test.go (5 tests)
• template_security_test.go (2 tests)
• markdown_test.go (2 tests)
• output_test.go (2 tests)
• version_test.go (2 tests)
• config_test.go (2 tests)

Recommendation: Can share DB or may not need DB at all

Migration Tests:

• migrate_test.go (3 tests)
• migrate_hash_ids_test.go (4 tests)
• repair_deps_test.go (4 tests)

Recommendation: Need isolation (modify DB schema)

Doctor Tests:

• doctor_test.go (13 tests)
• doctor/legacy_test.go tests

Recommendation: Mix - some can share, some need isolation

Misc Tests:

• ✅ compact_test.go (10 tests → 1 suite + 4 standalone = Phase 2 DONE)
• duplicates_test.go (5 tests)
• epic_test.go (3 tests)
• hooks_test.go (6 tests)
• info_test.go (5 tests)
• nodb_test.go (6 tests)
• restore_test.go (6 tests)
• worktree_test.go (2 tests)
• scripttest_test.go (1 test)
• direct_mode_test.go (1 test)
• autostart_test.go (3 tests)
• autoimport_test.go (9 tests)
• deletion_tracking_test.go (12 tests)
• rename_prefix_test.go (3 tests)
• rename_prefix_repair_test.go (1 test)
• status_test.go (3 tests)
• sync_merge_test.go (4 tests)
• jsonl_integrity_test.go (2 tests)
• export_staleness_test.go (5 tests)
• export_integrity_integration_test.go (3 tests)
• flush_manager_test.go (12 tests)
• daemon_debouncer_test.go (8 tests)
• daemon_rotation_test.go (4 tests)
• daemons_test.go (2 tests)
• daemon_watcher_platform_test.go (3 tests)
• helpers_test.go (4 tests)

Proposed Refactoring Plan

Phase 1: High Priority (P1) - Quick Wins ✓ COMPLETE

All P1 files refactored for immediate speedup:

1. ✓ create_test.go (bd-y6d) - Template refactor → TestCreateSuite
2. ✓ dep_test.go - Dependency tests → TestDependencySuite
3. ✓ stale_test.go - Stale issue tests → Individual test functions (data isolation required)
4. ✓ comments_test.go - Comment tests → TestCommentsSuite
5. ✓ list_test.go - List/search tests → TestListCommandSuite + TestListQueryCapabilitiesSuite
6. ✓ ready_test.go - Ready state tests → TestReadySuite

Results: All P1 tests now run in 0.43 seconds (vs. estimated 10+ minutes before)

Pattern to follow: Use label_test.go as the template!

func TestCreateSuite(t *testing.T) {
    tmpDir := t.TempDir()
    testDB := filepath.Join(tmpDir, ".beads", "beads.db")
    s := newTestStore(t, testDB)
    ctx := context.Background()

    t.Run("BasicIssue", func(t *testing.T) { /* test */ })
    t.Run("WithDescription", func(t *testing.T) { /* test */ })
    // ... etc
}

Phase 2: Medium Priority (P2) - Moderate Gains

After Phase 1 success:

1. main_test.go - Audit for DB-only vs CLI tests
2. integrity_test.go - Many helper calls, big win
3. export_import_test.go - Already has helper pattern

Phase 3: Special Cases (P3) - Complex Refactors

Handle tests that need mixed isolation:

1. Review daemon tests for DB-only portions
2. Review CLI tests for unit-testable logic
3. Consider utility functions that don't need DB

Success Metrics

Before (Current):

• 279-280 tests
• Each with newTestStore() = 280 DB initializations
• Estimated time: 8+ minutes

After (Proposed):

• 10-15 test suites for DB tests = ~15 DB initializations
• ~65 isolated tests (daemon, git, filesystem) = ~65 DB initializations
• Total: ~80 DB initializations (down from 280)
• Expected time: 1-2 minutes (5-8x speedup)

Per-Suite Expectations:

Suite	Current	Proposed	Speedup
TestCreateSuite	11 DBs	1 DB	10x
TestDependencySuite	4 DBs	1 DB	4x
TestStaleSuite	5 DBs	1 DB	5x
TestIntegritySuite	15 DBs	1 DB	15x
TestMainSuite	14 DBs	1-2 DBs	7-14x

Implementation Strategy

1. Use label_test.go as template - It already shows the pattern!
2. Start with create_test.go (bd-y6d) - Clear, simple, 11 tests
3. Validate speedup - Measure before/after for confidence
4. Apply pattern to other P1 files
5. Document pattern in test_helpers_test.go for future tests

Key Insights

1. ~150 tests can immediately benefit from shared DB setup
2. ~65 tests need isolation (daemon, git, filesystem)
3. ~65 tests need analysis (mixed or may not need DB)
4. label_test.go shows the ideal pattern - use it as the template!
5. Primary bottleneck: Repeated newTestStore() calls
6. Quick wins: Files with 5+ tests using newTestStore()

Next Steps

1. ✓ Complete this audit (bd-c49)
2. ✓ Refactor create_test.go (bd-y6d) using label_test.go pattern
3. ✓ Measure and validate speedup
4. ✓ Apply to remaining P1 files (bd-1rh)
5. → Tackle P2 files (main_test.go, integrity_test.go, export_import_test.go)
6. → Document best practices

Phase 1 Completion Summary (bd-1rh)

Status: ✓ COMPLETE - All 6 P1 test files refactored Runtime: 0.43 seconds for all P1 tests Speedup: Estimated 10-20x improvement Goal: Under 2 minutes for full test suite after all phases - ON TRACK

Key Learnings:

1. Shared DB pattern works well for most pure DB tests
2. Data pollution can occur when tests create overlapping data (e.g., stale_test.go)
3. Solution for pollution: Either use unique ID prefixes per subtest OR split into separate suites
4. ID prefix validation requires test IDs to match "test-*" pattern
5. SQLite datetime functions needed for timestamp manipulation in tests