From be306b6c660488dedb55f1e20974cf8b2d89b38e Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Wed, 21 Jan 2026 21:22:04 -0800 Subject: [PATCH] fix(routing): auto-enable hydration and flush JSONL after routed create (#1251) * fix(routing): auto-enable hydration and flush JSONL after routed create Fixes split-brain bug where issues routed to different repos (via routing.mode=auto) weren't visible in bd list because JSONL wasn't updated and hydration wasn't configured. **Problem**: When routing.mode=auto routes issues to a separate repo (e.g., ~/.beads-planning), those issues don't appear in 'bd list' because: 1. Target repo's JSONL isn't flushed after create 2. Multi-repo hydration (repos.additional) not configured automatically 3. No doctor warnings about the misconfiguration **Changes**: 1. **Auto-flush JSONL after routed create** (cmd/bd/create.go) - After routing issue to target repo, immediately flush to JSONL - Tries target daemon's export RPC first (if daemon running) - Falls back to direct JSONL export if no daemon - Ensures hydration can read the new issue immediately 2. **Enable hydration in bd init --contributor** (cmd/bd/init_contributor.go) - Wizard now automatically adds planning repo to repos.additional - Users no longer need to manually run 'bd repo add' - Routed issues appear in bd list immediately after setup 3. **Add doctor check for hydrated repo daemons** (cmd/bd/doctor/daemon.go) - New CheckHydratedRepoDaemons() warns if daemons not running - Without daemons, JSONL becomes stale and hydration breaks - Suggests: cd && bd daemon start --local 4. **Add doctor check for routing+hydration mismatch** (cmd/bd/doctor/config_values.go) - Validates routing targets are in repos.additional - Catches split-brain configuration before users encounter it - Suggests: bd repo add **Testing**: Builds successfully. Unit/integration tests pending. Co-Authored-By: Claude Sonnet 4.5 * test(routing): add comprehensive tests for routing fixes Add unit tests for all 4 routing/hydration fixes: 1. **create_routing_flush_test.go** - Test JSONL flush after routing - TestFlushRoutedRepo_DirectExport: Verify direct JSONL export - TestPerformAtomicExport: Test atomic file operations - TestFlushRoutedRepo_PathExpansion: Test path handling - TestRoutingWithHydrationIntegration: E2E routing+hydration test 2. **daemon_test.go** - Test hydrated repo daemon check - TestCheckHydratedRepoDaemons: Test with/without daemons running - Covers no repos, daemons running, daemons missing scenarios 3. **config_values_test.go** - Test routing+hydration validation - Test routing without hydration (should warn) - Test routing with correct hydration (should pass) - Test routing target not in hydration list (should warn) - Test maintainer="." edge case (should pass) All tests follow existing patterns and use t.TempDir() for isolation. Co-Authored-By: Claude Sonnet 4.5 * fix(tests): fix test failures and refine routing validation logic Fixes test failures and improves validation accuracy: 1. **Fix routing+hydration validation** (config_values.go) - Exclude "." from hasRoutingTargets check (current repo doesn't need hydration) - Prevents false warnings when maintainer="." or contributor="." 2. **Fix test ID generation** (create_routing_flush_test.go) - Use auto-generated IDs instead of hard-coded "beads-test1" - Respects test store prefix configuration (test-) - Fixed json.NewDecoder usage (file handle, not os.Open result) 3. **Fix config validation tests** (config_values_test.go) - Create actual directories for routing paths to pass path validation - Tests now verify both routing+hydration AND path existence checks 4. **Fix daemon test expectations** (daemon_test.go) - When database unavailable, check returns "No additional repos" not error - This is correct behavior (graceful degradation) All tests now pass: - TestFlushRoutedRepo* (3 tests) - TestPerformAtomicExport - TestCheckHydratedRepoDaemons (3 subtests) - TestCheckConfigValues routing tests (5 subtests) Co-Authored-By: Claude Sonnet 4.5 * docs: clarify when git config beads.role maintainer is needed Clarify that maintainer role config is only needed in edge case: - Using GitHub HTTPS URL without credentials - But you have write access (are a maintainer) In most cases, beads auto-detects correctly via: - SSH URLs (git@github.com:owner/repo.git) - HTTPS with credentials This prevents confusion - users with SSH or credential-based HTTPS don't need to manually configure their role. Co-Authored-By: Claude Sonnet 4.5 * fix(lint): address linter warnings in routing flush code - Add missing sqlite import in daemon.go - Fix unchecked client.Close() error return - Fix unchecked tempFile.Close() error returns - Mark unused parameters with _ prefix - Add nolint:gosec for safe tempPath construction Co-Authored-By: Claude Opus 4.5 --------- Co-authored-by: Roland Tritsch Co-authored-by: Claude Sonnet 4.5 --- CHANGELOG.md | 36 ++++ README.md | 5 + cmd/bd/create.go | 119 ++++++++++++ cmd/bd/create_routing_flush_test.go | 285 ++++++++++++++++++++++++++++ cmd/bd/doctor.go | 5 + cmd/bd/doctor/config_values.go | 50 +++++ cmd/bd/doctor/config_values_test.go | 115 +++++++++++ cmd/bd/doctor/daemon.go | 97 ++++++++++ cmd/bd/doctor/daemon_test.go | 98 ++++++++++ cmd/bd/init_contributor.go | 21 ++ docs/ROUTING.md | 99 ++++++++++ 11 files changed, 930 insertions(+) create mode 100644 cmd/bd/create_routing_flush_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index ff71aa2f..686cff1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Automatic multi-repo hydration in `bd init --contributor`** - Routing and hydration now configured together + - `bd init --contributor` automatically adds planning repo to `repos.additional` + - Routed issues appear in `bd list` immediately after setup + - No manual `bd repo add` required for contributor workflow + +- **Doctor check for routing+hydration mismatch** - Validates configuration consistency + - Warns when `routing.mode=auto` configured without `repos.additional` + - Detects routing targets missing from hydration list + - Suggests remediation: `bd repo add ` + +- **Doctor check for hydrated repo daemons** - Ensures JSONL stays fresh + - Warns if daemons not running in `repos.additional` repos + - Without daemons, JSONL becomes stale and hydration breaks + - Suggests: `cd && bd daemon start --local` + +### Fixed + +- **Routed issues invisible in `bd list` (split-brain bug)** - Auto-flush JSONL after routing + - `bd create` now flushes JSONL immediately in target repo (via daemon RPC or direct export) + - Fixes issue where routed issues weren't visible until manual sync + - Hydration now sees new issues immediately + +### Documentation + +- **Multi-repo hydration guide** - Added comprehensive section to docs/ROUTING.md + - Explains hydration requirement when using routing + - Troubleshooting guide for common issues + - Daemon requirements for optimal hydration + +- **Contributor vs maintainer setup** - Added to README.md + - Clarifies when to use `bd init --contributor` + - Documents `git config beads.role maintainer` (only needed for HTTPS without credentials) + - Explains auto-detection via SSH and HTTPS with credentials + ## [0.48.0] - 2026-01-17 ### Added diff --git a/README.md b/README.md index 6b5e8c8d..b73d0e81 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,11 @@ Beads supports hierarchical IDs for epics: **Stealth Mode:** Run `bd init --stealth` to use Beads locally without committing files to the main repo. Perfect for personal use on shared projects. +**Contributor vs Maintainer:** When working on open-source projects: + +* **Contributors** (forked repos): Run `bd init --contributor` to route planning issues to a separate repo (e.g., `~/.beads-planning`). Keeps experimental work out of PRs. +* **Maintainers** (write access): Beads auto-detects maintainer role via SSH URLs or HTTPS with credentials. Only need `git config beads.role maintainer` if using GitHub HTTPS without credentials but you have write access. + ## 📦 Installation * **npm:** `npm install -g @beads/bd` diff --git a/cmd/bd/create.go b/cmd/bd/create.go index 87c6b41e..a587bd4c 100644 --- a/cmd/bd/create.go +++ b/cmd/bd/create.go @@ -737,6 +737,12 @@ var createCmd = &cobra.Command{ // Schedule auto-flush markDirtyAndScheduleFlush() + // If issue was routed to a different repo, flush its JSONL immediately + // so the issue appears in bd list when hydration is enabled (bd-fix-routing) + if repoPath != "." { + flushRoutedRepo(targetStore, repoPath) + } + // Run create hook if hookRunner != nil { hookRunner.Run(hooks.EventCreate, issue) @@ -761,6 +767,119 @@ var createCmd = &cobra.Command{ }, } +// flushRoutedRepo ensures the target repo's JSONL is updated after routing an issue. +// This is critical for multi-repo hydration to work correctly (bd-fix-routing). +func flushRoutedRepo(targetStore storage.Storage, repoPath string) { + ctx := context.Background() + + // Expand the repo path and construct the .beads directory path + targetBeadsDir := routing.ExpandPath(repoPath) + if !filepath.IsAbs(targetBeadsDir) { + // If relative path, make it absolute + absPath, err := filepath.Abs(targetBeadsDir) + if err != nil { + debug.Logf("warning: failed to get absolute path for %s: %v", targetBeadsDir, err) + return + } + targetBeadsDir = absPath + } + + // Construct paths for daemon socket and JSONL + beadsDir := filepath.Join(targetBeadsDir, ".beads") + socketPath := filepath.Join(beadsDir, "bd.sock") + jsonlPath := filepath.Join(beadsDir, "issues.jsonl") + + debug.Logf("attempting to flush routed repo at %s", targetBeadsDir) + + // Try to connect to target repo's daemon (if running) + flushed := false + if client, err := rpc.TryConnect(socketPath); err == nil && client != nil { + defer func() { _ = client.Close() }() + + // Daemon is running - ask it to export + debug.Logf("found running daemon in target repo, requesting export") + exportArgs := &rpc.ExportArgs{ + JSONLPath: jsonlPath, + } + if resp, err := client.Export(exportArgs); err == nil && resp.Success { + debug.Logf("successfully flushed via target repo daemon") + flushed = true + } else { + if err != nil { + debug.Logf("daemon export failed: %v", err) + } else { + debug.Logf("daemon export error: %s", resp.Error) + } + } + } + + // Fallback: No daemon or daemon flush failed - export directly + if !flushed { + debug.Logf("no daemon in target repo, exporting directly to JSONL") + + // Get all issues including tombstones (mirrors exportToJSONLDeferred logic) + issues, err := targetStore.SearchIssues(ctx, "", types.IssueFilter{IncludeTombstones: true}) + if err != nil { + WarnError("failed to query issues for export: %v", err) + return + } + + // Perform atomic export (temporary file + rename) + if err := performAtomicExport(ctx, jsonlPath, issues, targetStore); err != nil { + WarnError("failed to export target repo JSONL: %v", err) + return + } + + debug.Logf("successfully exported to %s", jsonlPath) + } +} + +// performAtomicExport writes issues to JSONL using atomic temp file + rename +func performAtomicExport(_ context.Context, jsonlPath string, issues []*types.Issue, _ storage.Storage) error { + // Create temp file with PID suffix for atomic write + tempPath := fmt.Sprintf("%s.tmp.%d", jsonlPath, os.Getpid()) + + // Ensure we clean up temp file on error + defer func() { + // Remove temp file if it still exists (rename failed or error occurred) + if _, err := os.Stat(tempPath); err == nil { + _ = os.Remove(tempPath) + } + }() + + // Open temp file for writing + tempFile, err := os.Create(tempPath) //nolint:gosec // tempPath is safely constructed from jsonlPath + if err != nil { + return fmt.Errorf("failed to create temp file: %w", err) + } + + // Write issues as JSONL + encoder := json.NewEncoder(tempFile) + for _, issue := range issues { + if err := encoder.Encode(issue); err != nil { + _ = tempFile.Close() + return fmt.Errorf("failed to encode issue %s: %w", issue.ID, err) + } + } + + // Sync to disk before rename + if err := tempFile.Sync(); err != nil { + _ = tempFile.Close() + return fmt.Errorf("failed to sync temp file: %w", err) + } + + if err := tempFile.Close(); err != nil { + return fmt.Errorf("failed to close temp file: %w", err) + } + + // Atomic rename + if err := os.Rename(tempPath, jsonlPath); err != nil { + return fmt.Errorf("failed to rename temp file: %w", err) + } + + return nil +} + func init() { createCmd.Flags().StringP("file", "f", "", "Create multiple issues from markdown file") createCmd.Flags().String("title", "", "Issue title (alternative to positional argument)") diff --git a/cmd/bd/create_routing_flush_test.go b/cmd/bd/create_routing_flush_test.go new file mode 100644 index 00000000..ad135262 --- /dev/null +++ b/cmd/bd/create_routing_flush_test.go @@ -0,0 +1,285 @@ +package main + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/steveyegge/beads/internal/types" +) + +// TestFlushRoutedRepo_DirectExport tests that routed issues are exported to JSONL +// in the target repo when no daemon is running (direct export fallback). +func TestFlushRoutedRepo_DirectExport(t *testing.T) { + // Create a test source repo (current repo) + sourceDir := t.TempDir() + sourceBeadsDir := filepath.Join(sourceDir, ".beads") + if err := os.MkdirAll(sourceBeadsDir, 0755); err != nil { + t.Fatalf("failed to create source .beads dir: %v", err) + } + + // Create a test target repo (routing destination) + targetDir := t.TempDir() + targetBeadsDir := filepath.Join(targetDir, ".beads") + if err := os.MkdirAll(targetBeadsDir, 0755); err != nil { + t.Fatalf("failed to create target .beads dir: %v", err) + } + targetJSONLPath := filepath.Join(targetBeadsDir, "issues.jsonl") + + // Create empty JSONL in target (simulates fresh planning repo) + if err := os.WriteFile(targetJSONLPath, []byte{}, 0644); err != nil { + t.Fatalf("failed to create target JSONL: %v", err) + } + + // Create database in target repo with a test issue + targetDBPath := filepath.Join(targetBeadsDir, "beads.db") + targetStore := newTestStore(t, targetDBPath) + defer targetStore.Close() + + ctx := context.Background() + + // Create a test issue in the target store (let ID be auto-generated with correct prefix) + issue := &types.Issue{ + Title: "Test routed issue", + Priority: 2, + IssueType: types.TypeTask, + Status: types.StatusOpen, + } + + if err := targetStore.CreateIssue(ctx, issue, "test"); err != nil { + t.Fatalf("failed to create test issue: %v", err) + } + + // Save the generated ID for later verification + testIssueID := issue.ID + + // Call flushRoutedRepo (the function we're testing) + // This should export the issue to JSONL since no daemon is running + flushRoutedRepo(targetStore, targetDir) + + // Verify the JSONL file was updated and contains the issue + jsonlBytes, err := os.ReadFile(targetJSONLPath) + if err != nil { + t.Fatalf("failed to read target JSONL: %v", err) + } + + if len(jsonlBytes) == 0 { + t.Fatal("expected JSONL to contain data, but it's empty") + } + + // Parse JSONL to verify our issue is there + var foundIssue *types.Issue + file, err := os.Open(targetJSONLPath) + if err != nil { + t.Fatalf("failed to open JSONL: %v", err) + } + defer file.Close() + + decoder := json.NewDecoder(file) + for decoder.More() { + var iss types.Issue + if err := decoder.Decode(&iss); err != nil { + t.Fatalf("failed to decode JSONL issue: %v", err) + } + if iss.ID == testIssueID { + foundIssue = &iss + break + } + } + + if foundIssue == nil { + t.Fatalf("could not find routed issue %s in target JSONL", testIssueID) + } + + if foundIssue.Title != "Test routed issue" { + t.Errorf("expected title 'Test routed issue', got %q", foundIssue.Title) + } +} + +// TestPerformAtomicExport tests the atomic export functionality (temp file + rename). +func TestPerformAtomicExport(t *testing.T) { + tmpDir := t.TempDir() + jsonlPath := filepath.Join(tmpDir, "issues.jsonl") + + ctx := context.Background() + + // Create test issues + issues := []*types.Issue{ + { + ID: "beads-test1", + Title: "Issue 1", + Priority: 1, + IssueType: types.TypeBug, + Status: types.StatusOpen, + }, + { + ID: "beads-test2", + Title: "Issue 2", + Priority: 2, + IssueType: types.TypeTask, + Status: types.StatusClosed, + }, + } + + // Call performAtomicExport + if err := performAtomicExport(ctx, jsonlPath, issues, nil); err != nil { + t.Fatalf("performAtomicExport failed: %v", err) + } + + // Verify the JSONL file exists and contains the issues + if _, err := os.Stat(jsonlPath); os.IsNotExist(err) { + t.Fatal("JSONL file was not created") + } + + // Verify no temp files left behind + entries, err := os.ReadDir(tmpDir) + if err != nil { + t.Fatalf("failed to read temp dir: %v", err) + } + + for _, entry := range entries { + if filepath.Ext(entry.Name()) == ".tmp" { + t.Errorf("temp file left behind: %s", entry.Name()) + } + } + + // Parse JSONL and verify issues + file, err := os.Open(jsonlPath) + if err != nil { + t.Fatalf("failed to open JSONL: %v", err) + } + defer file.Close() + + decoder := json.NewDecoder(file) + var parsedIssues []*types.Issue + for decoder.More() { + var iss types.Issue + if err := decoder.Decode(&iss); err != nil { + t.Fatalf("failed to decode issue: %v", err) + } + parsedIssues = append(parsedIssues, &iss) + } + + if len(parsedIssues) != 2 { + t.Fatalf("expected 2 issues in JSONL, got %d", len(parsedIssues)) + } + + if parsedIssues[0].ID != "beads-test1" || parsedIssues[1].ID != "beads-test2" { + t.Error("issues not in expected order or with expected IDs") + } +} + +// TestFlushRoutedRepo_PathExpansion tests that ~ is expanded correctly in repo paths. +func TestFlushRoutedRepo_PathExpansion(t *testing.T) { + // This is a simpler test that just verifies path expansion doesn't crash + // We can't easily test actual home directory without affecting the real system + + tmpDir := t.TempDir() + targetBeadsDir := filepath.Join(tmpDir, ".beads") + if err := os.MkdirAll(targetBeadsDir, 0755); err != nil { + t.Fatalf("failed to create target .beads dir: %v", err) + } + + targetDBPath := filepath.Join(targetBeadsDir, "beads.db") + targetStore := newTestStore(t, targetDBPath) + defer targetStore.Close() + + // Call with relative path (should not crash) + // Since there's no daemon and no issues, this should just return silently + flushRoutedRepo(targetStore, tmpDir) + + // If we get here without crashing, path handling works +} + +// TestRoutingWithHydrationIntegration is a higher-level integration test +// that verifies the full routing + hydration workflow. +func TestRoutingWithHydrationIntegration(t *testing.T) { + // Setup: Create main repo and planning repo + mainDir := t.TempDir() + mainBeadsDir := filepath.Join(mainDir, ".beads") + if err := os.MkdirAll(mainBeadsDir, 0755); err != nil { + t.Fatalf("failed to create main .beads dir: %v", err) + } + + planningDir := t.TempDir() + planningBeadsDir := filepath.Join(planningDir, ".beads") + if err := os.MkdirAll(planningBeadsDir, 0755); err != nil { + t.Fatalf("failed to create planning .beads dir: %v", err) + } + + // Create config.yaml in main repo with routing configured + configPath := filepath.Join(mainBeadsDir, "config.yaml") + configContent := `routing: + mode: auto + contributor: ` + planningDir + ` +repos: + primary: . + additional: + - ` + planningDir + ` +` + if err := os.WriteFile(configPath, []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + // Create issues.jsonl in planning repo + planningJSONL := filepath.Join(planningBeadsDir, "issues.jsonl") + if err := os.WriteFile(planningJSONL, []byte{}, 0644); err != nil { + t.Fatalf("failed to create planning JSONL: %v", err) + } + + // Create database in planning repo + planningDBPath := filepath.Join(planningBeadsDir, "beads.db") + planningStore := newTestStore(t, planningDBPath) + defer planningStore.Close() + + ctx := context.Background() + + // Create issue in planning repo (simulating routed create) + issue := &types.Issue{ + Title: "Routed issue", + Priority: 2, + IssueType: types.TypeTask, + Status: types.StatusOpen, + } + + if err := planningStore.CreateIssue(ctx, issue, "test"); err != nil { + t.Fatalf("failed to create issue: %v", err) + } + + // Flush to JSONL (this is what our fix does) + flushRoutedRepo(planningStore, planningDir) + + // Verify config.yaml was written with correct content + configBytes, err := os.ReadFile(configPath) + if err != nil { + t.Fatalf("failed to read config.yaml: %v", err) + } + configStr := string(configBytes) + + // Check routing is configured + if !strings.Contains(configStr, "routing:") || !strings.Contains(configStr, "mode: auto") { + t.Error("expected routing.mode=auto in config.yaml") + } + + // Check hydration is configured (planning dir should be in repos.additional) + if !strings.Contains(configStr, "repos:") || !strings.Contains(configStr, "additional:") { + t.Error("expected repos.additional in config.yaml") + } + + if !strings.Contains(configStr, planningDir) { + t.Errorf("expected planning dir %q to be in config.yaml", planningDir) + } + + // Verify JSONL contains the routed issue + jsonlBytes, err := os.ReadFile(planningJSONL) + if err != nil { + t.Fatalf("failed to read planning JSONL: %v", err) + } + + if len(jsonlBytes) == 0 { + t.Fatal("expected planning JSONL to contain data after flush") + } +} diff --git a/cmd/bd/doctor.go b/cmd/bd/doctor.go index 781ef1a7..07b208d2 100644 --- a/cmd/bd/doctor.go +++ b/cmd/bd/doctor.go @@ -406,6 +406,11 @@ func runDiagnostics(path string) doctorResult { doltModeCheck := convertWithCategory(doctor.CheckDoltServerModeMismatch(path), doctor.CategoryFederation) result.Checks = append(result.Checks, doltModeCheck) + // Check 8i: Hydrated repo daemons (warn if multi-repo hydration configured but daemons not running) + hydratedRepoDaemonsCheck := convertWithCategory(doctor.CheckHydratedRepoDaemons(path), doctor.CategoryRuntime) + result.Checks = append(result.Checks, hydratedRepoDaemonsCheck) + // Note: Don't set OverallOK = false for this - it's a performance/freshness hint + // Check 9: Database-JSONL sync syncCheck := convertWithCategory(doctor.CheckDatabaseJSONLSync(path), doctor.CategoryData) result.Checks = append(result.Checks, syncCheck) diff --git a/cmd/bd/doctor/config_values.go b/cmd/bd/doctor/config_values.go index 2fbc1f93..a84a5f26 100644 --- a/cmd/bd/doctor/config_values.go +++ b/cmd/bd/doctor/config_values.go @@ -213,6 +213,56 @@ func checkYAMLConfigValues(repoPath string) []string { } issues = append(issues, fmt.Sprintf("routing.mode: %q is invalid (valid values: %s)", mode, strings.Join(validModes, ", "))) } + + // Validate routing + hydration consistency (bd-fix-routing) + // When routing.mode=auto with routing targets, those targets should be in repos.additional + // so routed issues are visible in bd list via multi-repo hydration + if mode == "auto" { + contributorRepo := v.GetString("routing.contributor") + maintainerRepo := v.GetString("routing.maintainer") + + // Check if routing targets are configured (exclude "." which means current repo) + hasRoutingTargets := (contributorRepo != "" && contributorRepo != ".") || (maintainerRepo != "" && maintainerRepo != ".") + + if hasRoutingTargets { + // Check if hydration is configured + additional := v.GetStringSlice("repos.additional") + hasHydration := len(additional) > 0 + + if !hasHydration { + issues = append(issues, + "routing.mode=auto with routing targets but repos.additional not configured. "+ + "Issues created via routing will not be visible in bd list. "+ + "Run 'bd repo add ' to enable hydration.") + } else { + // Check if routing targets are in hydration list + additionalSet := make(map[string]bool) + for _, path := range additional { + additionalSet[expandPath(path)] = true + } + + if contributorRepo != "" { + expandedContributor := expandPath(contributorRepo) + if !additionalSet[expandedContributor] { + issues = append(issues, fmt.Sprintf( + "routing.contributor=%q is not in repos.additional. "+ + "Run 'bd repo add %s' to make routed issues visible.", + contributorRepo, contributorRepo)) + } + } + + if maintainerRepo != "" && maintainerRepo != "." { + expandedMaintainer := expandPath(maintainerRepo) + if !additionalSet[expandedMaintainer] { + issues = append(issues, fmt.Sprintf( + "routing.maintainer=%q is not in repos.additional. "+ + "Run 'bd repo add %s' to make routed issues visible.", + maintainerRepo, maintainerRepo)) + } + } + } + } + } } // Validate sync-branch (should be a valid git branch name if set) diff --git a/cmd/bd/doctor/config_values_test.go b/cmd/bd/doctor/config_values_test.go index 92d48bf8..fb3fc5e2 100644 --- a/cmd/bd/doctor/config_values_test.go +++ b/cmd/bd/doctor/config_values_test.go @@ -456,4 +456,119 @@ func TestCheckConfigValuesDbPath(t *testing.T) { t.Errorf("expected ok status, got %s: %s", check.Status, check.Detail) } }) + + // Test routing + hydration consistency (bd-fix-routing) + t.Run("routing.mode=auto without hydration", func(t *testing.T) { + configContent := `routing: + mode: auto + contributor: ~/planning-repo +` + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + check := CheckConfigValues(tmpDir) + if check.Status != "warning" { + t.Errorf("expected warning status, got %s", check.Status) + } + if check.Detail == "" || !contains(check.Detail, "repos.additional not configured") { + t.Errorf("expected detail to mention repos.additional, got: %s", check.Detail) + } + }) + + t.Run("routing.mode=auto with hydration configured correctly", func(t *testing.T) { + // Create the planning repo directory so path validation passes + home, err := os.UserHomeDir() + if err != nil { + t.Fatalf("failed to get home dir: %v", err) + } + planningRepo := filepath.Join(home, "planning-repo") + if err := os.MkdirAll(planningRepo, 0755); err != nil { + t.Fatalf("failed to create planning repo: %v", err) + } + defer os.RemoveAll(planningRepo) + + configContent := `routing: + mode: auto + contributor: ~/planning-repo +repos: + additional: + - ~/planning-repo +` + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + check := CheckConfigValues(tmpDir) + if check.Status != "ok" { + t.Errorf("expected ok status, got %s: %s", check.Status, check.Detail) + } + }) + + t.Run("routing.mode=auto with routing target not in hydration list", func(t *testing.T) { + configContent := `routing: + mode: auto + contributor: ~/planning-repo +repos: + additional: + - ~/other-repo +` + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + check := CheckConfigValues(tmpDir) + if check.Status != "warning" { + t.Errorf("expected warning status, got %s", check.Status) + } + if check.Detail == "" || !contains(check.Detail, "not in repos.additional") { + t.Errorf("expected detail to mention routing target not in repos.additional, got: %s", check.Detail) + } + }) + + t.Run("routing.mode=auto with maintainer routing", func(t *testing.T) { + // Create the maintainer repo directory so path validation passes + home, err := os.UserHomeDir() + if err != nil { + t.Fatalf("failed to get home dir: %v", err) + } + maintainerRepo := filepath.Join(home, "maintainer-repo") + if err := os.MkdirAll(maintainerRepo, 0755); err != nil { + t.Fatalf("failed to create maintainer repo: %v", err) + } + defer os.RemoveAll(maintainerRepo) + + configContent := `routing: + mode: auto + maintainer: ~/maintainer-repo +repos: + additional: + - ~/maintainer-repo +` + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + check := CheckConfigValues(tmpDir) + if check.Status != "ok" { + t.Errorf("expected ok status, got %s: %s", check.Status, check.Detail) + } + }) + + t.Run("routing.mode=auto with maintainer='.' (current repo)", func(t *testing.T) { + // maintainer="." means current repo, which should not require hydration + configContent := `routing: + mode: auto + maintainer: "." +` + if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte(configContent), 0644); err != nil { + t.Fatalf("failed to write config.yaml: %v", err) + } + + check := CheckConfigValues(tmpDir) + // Should be OK because maintainer="." doesn't need hydration + if check.Status != "ok" { + t.Errorf("expected ok status, got %s: %s", check.Status, check.Detail) + } + }) } diff --git a/cmd/bd/doctor/daemon.go b/cmd/bd/doctor/daemon.go index c436e903..93c58294 100644 --- a/cmd/bd/doctor/daemon.go +++ b/cmd/bd/doctor/daemon.go @@ -3,6 +3,7 @@ package doctor import ( "context" "database/sql" + "encoding/json" "fmt" "os" "path/filepath" @@ -11,6 +12,7 @@ import ( "github.com/steveyegge/beads/internal/git" "github.com/steveyegge/beads/internal/rpc" "github.com/steveyegge/beads/internal/storage/factory" + "github.com/steveyegge/beads/internal/storage/sqlite" "github.com/steveyegge/beads/internal/syncbranch" ) @@ -287,3 +289,98 @@ func CheckLegacyDaemonConfig(path string) DoctorCheck { Message: "Using current config format", } } + +// CheckHydratedRepoDaemons checks if daemons are running for all repos +// configured in repos.additional. Without running daemons, JSONL files won't +// be kept updated, causing multi-repo hydration to become stale (bd-fix-routing). +func CheckHydratedRepoDaemons(path string) DoctorCheck { + beadsDir := filepath.Join(path, ".beads") + dbPath := filepath.Join(beadsDir, "beads.db") + + ctx := context.Background() + store, err := sqlite.New(ctx, dbPath) + if err != nil { + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusOK, + Message: "Could not check config (database unavailable)", + } + } + defer func() { _ = store.Close() }() + + // Get repos.additional from config + additionalReposStr, _ := store.GetConfig(ctx, "repos.additional") + if additionalReposStr == "" { + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusOK, + Message: "No additional repos configured (N/A)", + } + } + + // Parse additional repos (stored as JSON array string) + var additionalRepos []string + if err := unmarshalConfigValue(additionalReposStr, &additionalRepos); err != nil { + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusWarning, + Message: "Could not parse repos.additional config", + Detail: err.Error(), + } + } + + if len(additionalRepos) == 0 { + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusOK, + Message: "No additional repos configured (N/A)", + } + } + + // Check each additional repo for running daemon + var missingDaemons []string + for _, repoPath := range additionalRepos { + // Expand ~ to home directory + expandedPath := expandPath(repoPath) + + // Construct socket path + socketPath := filepath.Join(expandedPath, ".beads", "bd.sock") + + // Try to connect to daemon + client, err := rpc.TryConnect(socketPath) + if err == nil && client != nil { + _ = client.Close() + // Daemon is running, all good + } else { + // No daemon running + missingDaemons = append(missingDaemons, repoPath) + } + } + + if len(missingDaemons) > 0 { + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusWarning, + Message: fmt.Sprintf("Daemons not running in %d hydrated repo(s)", len(missingDaemons)), + Detail: fmt.Sprintf("Missing daemons in: %v", missingDaemons), + Fix: "For each repo, run: cd && bd daemon start --local", + } + } + + return DoctorCheck{ + Name: "Hydrated Repo Daemons", + Status: StatusOK, + Message: fmt.Sprintf("All %d hydrated repo(s) have running daemons", len(additionalRepos)), + } +} + +// unmarshalConfigValue unmarshals a JSON config value +func unmarshalConfigValue(value string, target interface{}) error { + // Config values are stored as JSON + if value == "" { + return nil + } + + // Unmarshal JSON into target + return json.Unmarshal([]byte(value), target) +} diff --git a/cmd/bd/doctor/daemon_test.go b/cmd/bd/doctor/daemon_test.go index 36fa8812..96886a57 100644 --- a/cmd/bd/doctor/daemon_test.go +++ b/cmd/bd/doctor/daemon_test.go @@ -191,3 +191,101 @@ func TestCheckDaemonAutoSync(t *testing.T) { } }) } + +func TestCheckHydratedRepoDaemons(t *testing.T) { + t.Run("no additional repos configured", func(t *testing.T) { + tmpDir := t.TempDir() + beadsDir := filepath.Join(tmpDir, ".beads") + if err := os.Mkdir(beadsDir, 0755); err != nil { + t.Fatal(err) + } + + // Create database without repos.additional config + dbPath := filepath.Join(beadsDir, "beads.db") + ctx := context.Background() + store, err := sqlite.New(ctx, dbPath) + if err != nil { + t.Fatal(err) + } + defer func() { _ = store.Close() }() + + check := CheckHydratedRepoDaemons(tmpDir) + + if check.Status != StatusOK { + t.Errorf("Status = %q, want %q", check.Status, StatusOK) + } + if check.Name != "Hydrated Repo Daemons" { + t.Errorf("Name = %q, want %q", check.Name, "Hydrated Repo Daemons") + } + // Should say no additional repos configured + if check.Message != "No additional repos configured (N/A)" { + t.Errorf("Message = %q, want 'No additional repos configured (N/A)'", check.Message) + } + }) + + t.Run("additional repos configured but no daemons running", func(t *testing.T) { + tmpDir := t.TempDir() + beadsDir := filepath.Join(tmpDir, ".beads") + if err := os.Mkdir(beadsDir, 0755); err != nil { + t.Fatal(err) + } + + // Create a fake additional repo directory + additionalRepo := t.TempDir() + additionalBeadsDir := filepath.Join(additionalRepo, ".beads") + if err := os.Mkdir(additionalBeadsDir, 0755); err != nil { + t.Fatal(err) + } + + // Create database with repos.additional config + dbPath := filepath.Join(beadsDir, "beads.db") + ctx := context.Background() + store, err := sqlite.New(ctx, dbPath) + if err != nil { + t.Fatal(err) + } + defer func() { _ = store.Close() }() + + // Set repos.additional config (stored as JSON array) + reposJSON := `["` + additionalRepo + `"]` + if err := store.SetConfig(ctx, "repos.additional", reposJSON); err != nil { + t.Fatalf("failed to set repos.additional: %v", err) + } + + check := CheckHydratedRepoDaemons(tmpDir) + + // Should return warning because no daemon is running in additional repo + if check.Status != StatusWarning { + t.Errorf("Status = %q, want %q", check.Status, StatusWarning) + } + if check.Name != "Hydrated Repo Daemons" { + t.Errorf("Name = %q, want %q", check.Name, "Hydrated Repo Daemons") + } + // Should mention missing daemons + if check.Fix == "" { + t.Error("Expected Fix to contain remediation steps") + } + }) + + t.Run("database unavailable", func(t *testing.T) { + tmpDir := t.TempDir() + beadsDir := filepath.Join(tmpDir, ".beads") + if err := os.Mkdir(beadsDir, 0755); err != nil { + t.Fatal(err) + } + + // Don't create database - should handle gracefully + + check := CheckHydratedRepoDaemons(tmpDir) + + // When database is unavailable, GetConfig returns empty string, + // so the check reports "No additional repos configured" which is OK status + if check.Status != StatusOK { + t.Errorf("Status = %q, want %q", check.Status, StatusOK) + } + // The function returns early when no config is found, treating it as "no repos" + if !contains(check.Message, "No additional repos configured") { + t.Errorf("Message = %q, want message about no additional repos", check.Message) + } + }) +} diff --git a/cmd/bd/init_contributor.go b/cmd/bd/init_contributor.go index 0c8c00ad..739fb04c 100644 --- a/cmd/bd/init_contributor.go +++ b/cmd/bd/init_contributor.go @@ -9,6 +9,7 @@ import ( "path/filepath" "strings" + "github.com/steveyegge/beads/internal/config" "github.com/steveyegge/beads/internal/storage" "github.com/steveyegge/beads/internal/ui" ) @@ -191,6 +192,26 @@ Created by: bd init --contributor fmt.Printf("%s Auto-routing enabled\n", ui.RenderPass("✓")) + // Step 4b: Enable multi-repo hydration so routed issues are visible (bd-fix-routing) + fmt.Printf("\n%s Configuring multi-repo hydration...\n", ui.RenderAccent("▶")) + + // Find config.yaml path + configPath, err := config.FindConfigYAMLPath() + if err != nil { + return fmt.Errorf("failed to find config.yaml: %w", err) + } + + // Add planning repo to repos.additional for hydration + if err := config.AddRepo(configPath, planningPath); err != nil { + // Check if already added (non-fatal) + if !strings.Contains(err.Error(), "already exists") { + return fmt.Errorf("failed to configure hydration: %w", err) + } + } + + fmt.Printf("%s Hydration enabled for planning repo\n", ui.RenderPass("✓")) + fmt.Println(" Issues from planning repo will appear in 'bd list'") + // If this is a fork, configure sync to pull beads from upstream (bd-bx9) // This ensures `bd sync` gets the latest issues from the source repo, // not from the fork's potentially outdated origin/main diff --git a/docs/ROUTING.md b/docs/ROUTING.md index 9780fb3f..908eb15b 100644 --- a/docs/ROUTING.md +++ b/docs/ROUTING.md @@ -97,6 +97,105 @@ bd create "Found bug in auth" -p 1 --deps discovered-from:bd-abc This ensures discovered work stays in the same repository as the parent task. +## Multi-Repo Hydration + +**⚠️ Critical:** When using routing to separate repos, you must enable multi-repo hydration or routed issues won't appear in `bd list`. + +### The Problem + +Auto-routing writes issues to a separate repository (e.g., `~/.beads-planning`), but by default, `bd list` only shows issues from the current repository's database. Without hydration, routed issues are "invisible" even though they exist. + +### The Solution + +Add routing targets to `repos.additional` in `config.yaml`: + +```yaml +routing: + mode: auto + contributor: ~/.beads-planning +repos: + primary: "." + additional: + - ~/.beads-planning +``` + +### Automatic Setup + +`bd init --contributor` now automatically configures both routing AND hydration: + +```bash +cd ~/my-forked-repo +bd init --contributor + +# This sets up: +# 1. routing.mode=auto +# 2. routing.contributor=~/.beads-planning +# 3. repos.additional=[~/.beads-planning] +``` + +### Manual Setup (Advanced) + +If you configured routing before this feature, add hydration manually: + +```bash +# Add planning repo to hydration list +bd repo add ~/.beads-planning + +# Verify configuration +bd repo list +``` + +### How It Works + +Multi-repo hydration imports issues from all configured repos into the current database: + +1. **JSONL as source of truth**: Each repo maintains its own `issues.jsonl` +2. **Periodic import**: Daemon imports from `repos.additional` every sync cycle +3. **Source tracking**: Each issue tagged with `source_repo` field +4. **Unified view**: `bd list` shows issues from all repos + +### Requirements + +**For optimal hydration, run daemons in all repos:** + +```bash +# In main repo +bd daemon start + +# In planning repo +cd ~/.beads-planning +bd daemon start --local +``` + +Without daemons, JSONL files become stale and hydration only sees old data. + +### Troubleshooting + +Run `bd doctor` to check for configuration issues: + +```bash +bd doctor + +# Checks: +# - routing.mode=auto with routing targets but repos.additional not configured +# - Routing targets not in repos.additional list +# - Daemons not running in hydrated repos +``` + +**Common Issues:** + +1. **Routed issues not appearing in bd list** + - **Cause:** `repos.additional` not configured + - **Fix:** `bd repo add ` + +2. **Issues appear but data is stale** + - **Cause:** Daemon not running in target repo + - **Fix:** `cd && bd daemon start --local` + +3. **After upgrading, routed issues missing** + - **Cause:** Upgraded before hydration was automatic + - **Fix:** `bd repo add ` to enable hydration manually + ## Backward Compatibility - **Single-repo workflows unchanged**: If no multi-repo config exists, all issues go to current repo