docs: Add comprehensive documentation for version reporting changes
This commit is contained in:
matt wilkie
Steve Yegge
188
build-docs.md
Normal file
188
build-docs.md
Normal file
@@ -0,0 +1,188 @@
|
||||
# Noridoc: Build and Version Infrastructure
|
||||
|
||||
Path: @/ (root level - Makefile, .goreleaser.yml)
|
||||
|
||||
### Overview
|
||||
|
||||
The beads project uses a coordinated build and version reporting system that ensures all installation methods (direct `go install`, `make install`, GitHub releases, Homebrew, npm) produce binaries with complete version information including git commit hash and branch name.
|
||||
|
||||
This infrastructure is critical for debugging, auditing, and user support - it allows anyone to identify exactly what code their binary was built from.
|
||||
|
||||
### How it fits into the larger codebase
|
||||
|
||||
- **Build Entry Points**: The Makefile and .goreleaser.yml are the authoritative build configurations that users and CI/CD systems interact with. They control how version information flows into binaries.
|
||||
|
||||
- **Version Pipeline**: These files work with `@/cmd/bd/version.go` to establish the complete version reporting chain:
|
||||
- Build time: Extract git info via shell commands (Makefile) or goreleaser templates
|
||||
- Compilation: Pass info to Go compiler via `-X` ldflags
|
||||
- Runtime: Resolve functions in version.go retrieve and display the info
|
||||
|
||||
- **Installation Methods**: The build configuration enables multiple installation paths while maintaining version consistency:
|
||||
- `make install` - Used by developers building from source
|
||||
- `go install ./cmd/bd` - Direct Go installation with embedded ldflag injection
|
||||
- GitHub releases - Goreleaser-built binaries for all platforms
|
||||
- Homebrew - Pre-built binaries installed via `brew install bd`
|
||||
- npm - Node.js package that downloads pre-built binaries via postinstall hook
|
||||
- `./scripts/install.sh` - User-friendly build-from-source helper
|
||||
|
||||
- **Release Automation**: Goreleaser configuration integrates with GitHub Actions and the release process documented in `@/RELEASING.md`, ensuring released binaries have full version info.
|
||||
|
||||
### Core Implementation
|
||||
|
||||
**Makefile** (`@/Makefile`, lines 37-41 - `install` target):
|
||||
|
||||
The `install` target is the primary development build mechanism:
|
||||
|
||||
```makefile
|
||||
install:
|
||||
@echo "Installing bd to $$(go env GOPATH)/bin..."
|
||||
@bash -c 'commit=$$(git rev-parse HEAD 2>/dev/null || echo ""); \
|
||||
branch=$$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo ""); \
|
||||
go install -ldflags="-X main.Commit=$$commit -X main.Branch=$$branch" ./cmd/bd'
|
||||
```
|
||||
|
||||
How it works:
|
||||
1. Uses bash subshell to extract git information at install time
|
||||
2. `git rev-parse HEAD` gets the full commit hash
|
||||
3. `git rev-parse --abbrev-ref HEAD` gets the current branch name
|
||||
4. Passes both as ldflags to `go install` using the `-X` flag (variable assignment)
|
||||
5. The ldflags set `main.Commit` and `main.Branch` package variables
|
||||
6. These variables are then retrieved by functions in `@/cmd/bd/version.go` at runtime
|
||||
|
||||
Key implementation detail: The original target depended on `build`, but this was removed because `go install` is sufficient and handles compilation itself.
|
||||
|
||||
**Goreleaser Configuration** (`@/.goreleaser.yml`, lines 11-95):
|
||||
|
||||
Goreleaser builds binaries for all platforms and sets version information via ldflags. Each of the 5 build configurations uses identical ldflag patterns:
|
||||
|
||||
```yaml
|
||||
ldflags:
|
||||
- -s -w
|
||||
- -X main.Version={{.Version}}
|
||||
- -X main.Build={{.ShortCommit}}
|
||||
- -X main.Commit={{.Commit}}
|
||||
- -X main.Branch={{.Branch}}
|
||||
```
|
||||
|
||||
Platform configurations:
|
||||
1. **bd-linux-amd64** (lines 12-26): Linux 64-bit Intel
|
||||
2. **bd-linux-arm64** (lines 28-44): Linux 64-bit ARM (Apple Silicon support)
|
||||
3. **bd-darwin-amd64** (lines 46-60): macOS 64-bit Intel
|
||||
4. **bd-darwin-arm64** (lines 62-76): macOS 64-bit ARM (M1/M2/M3)
|
||||
5. **bd-windows-amd64** (lines 78-95): Windows 64-bit Intel with additional `-buildmode=exe` flag
|
||||
|
||||
The ldflags explained:
|
||||
- `-s -w`: Strip debug symbols to reduce binary size
|
||||
- `-X main.Version`: Semantic version (e.g., "0.29.0") from git tag
|
||||
- `-X main.Build`: Short commit hash for quick reference
|
||||
- `-X main.Commit`: Full commit hash for precise identification
|
||||
- `-X main.Branch`: Branch name for build context
|
||||
|
||||
Goreleaser template variables:
|
||||
- `{{.Version}}`: The release version from git tag
|
||||
- `{{.ShortCommit}}`: First 7 characters of commit (used for Build variable)
|
||||
- `{{.Commit}}`: Full commit hash
|
||||
- `{{.Branch}}`: Current git branch
|
||||
|
||||
**Installation Script** (`@/scripts/install.sh`):
|
||||
|
||||
Provides a user-friendly way to build from source with full version info:
|
||||
- Extracts git commit and branch
|
||||
- Calls `go install` with the same ldflags pattern as Makefile
|
||||
- Immediately verifies installation by running `bd version`
|
||||
|
||||
**Version Resolution Chain** (`@/cmd/bd/version.go`, lines 116-163):
|
||||
|
||||
The version.go file implements functions that retrieve the injected information:
|
||||
|
||||
1. **resolveCommitHash()** (lines 116-130):
|
||||
- First checks if `Commit` package variable was set via ldflag (most reliable)
|
||||
- Falls back to `runtime/debug.ReadBuildInfo()` to extract VCS info automatically embedded by `go build`
|
||||
- Returns empty string if neither source has data
|
||||
|
||||
2. **resolveBranch()** (lines 139-163):
|
||||
- First checks if `Branch` package variable was set via ldflag (most reliable)
|
||||
- Falls back to `runtime/debug.ReadBuildInfo()` to extract VCS branch info
|
||||
- Falls back to `git symbolic-ref --short HEAD` for runtime detection
|
||||
- Returns empty string if none available
|
||||
|
||||
3. **Output Formatting** (lines 52-58):
|
||||
- Displays commit and branch in human-readable format: `bd version 0.29.0 (dev: main@7e70940)`
|
||||
|
||||
### Things to Know
|
||||
|
||||
**Critical Design Decision - Why Explicit Ldflags**:
|
||||
|
||||
The Go toolchain (as of 1.18+) can automatically embed VCS information when compiling with `go build`, but this does NOT happen with `go install`. This creates an asymmetry:
|
||||
- `go build ./cmd/bd` → automatically embeds vcs.revision and vcs.branch
|
||||
- `go install ./cmd/bd` → does NOT embed VCS info automatically
|
||||
|
||||
The solution is to explicitly pass git information as ldflags in all build configurations. This ensures:
|
||||
- Users who run `make install` get full version info
|
||||
- Users who run `go install ./cmd/bd` need to explicitly set ldflags (via Makefile or script)
|
||||
- Released binaries from goreleaser have full version info (handled by goreleaser templates)
|
||||
- The version command is consistent regardless of installation method
|
||||
|
||||
**Issue #503 Root Cause**:
|
||||
|
||||
The original system relied on Go's automatic VCS embedding which only works with `go build`. When released binaries (built via goreleaser) or installed binaries (via `go install`) came without explicit ldflags, the `bd version` command couldn't report commit and branch information.
|
||||
|
||||
The fix adds explicit ldflag injection at all build points, creating a reliable pipeline independent of Go's automatic VCS embedding feature.
|
||||
|
||||
**Ldflag Variable Names**:
|
||||
|
||||
The variables in `@/cmd/bd/version.go` (lines 15-23) must match the ldflag paths in build configurations:
|
||||
- `main.Version` → Version variable
|
||||
- `main.Build` → Build variable
|
||||
- `main.Commit` → Commit variable
|
||||
- `main.Branch` → Branch variable
|
||||
|
||||
These are fully qualified with the package name (`main`) because the ldflag syntax is: `-X package.Variable=value`
|
||||
|
||||
**Build-Time vs Runtime Information**:
|
||||
|
||||
- **Build-Time** (injected via ldflags): Git commit and branch at the moment of compilation
|
||||
- Most reliable and consistent
|
||||
- Does not change after binary is created
|
||||
- Reflects the exact code in the binary
|
||||
|
||||
- **Runtime** (fallback via git commands): Current branch of the source directory
|
||||
- Used only if build-time info is not available
|
||||
- Can differ from build-time info (e.g., if working directory changes)
|
||||
- Useful for development but less reliable
|
||||
|
||||
**Release Process Integration**:
|
||||
|
||||
The build configuration integrates with `@/RELEASING.md`:
|
||||
1. Version tag is pushed to GitHub (e.g., `v0.29.0`)
|
||||
2. GitHub Actions/goreleaser automatically builds binaries for all platforms
|
||||
3. Goreleaser uses git metadata to populate `{{.Commit}}` and `{{.Branch}}` template variables
|
||||
4. Released binaries include full version info
|
||||
5. Users installing via any channel get consistent version reporting
|
||||
|
||||
**Testing Version Information**:
|
||||
|
||||
The test file `@/cmd/bd/version_test.go` includes:
|
||||
- `TestResolveCommitHash`: Verifies ldflag values are prioritized
|
||||
- `TestResolveBranch`: Verifies ldflag values are prioritized
|
||||
- `TestVersionOutputWithCommitAndBranch`: Verifies output formatting with real values
|
||||
|
||||
These tests simulate build-time injection by directly setting the package variables, ensuring the resolution chain works correctly.
|
||||
|
||||
**Multi-Platform Consistency**:
|
||||
|
||||
All 5 goreleaser build configurations use identical ldflag patterns. This ensures:
|
||||
- macOS (Intel and ARM) binaries have full version info
|
||||
- Linux (Intel and ARM) binaries have full version info
|
||||
- Windows binaries have full version info
|
||||
- Users on any platform have equal access to version information
|
||||
|
||||
**Dependency Requirements**:
|
||||
|
||||
- **Makefile**: Requires `git` and `bash` to be available at install time
|
||||
- **Goreleaser**: Requires git tags and repository metadata (automatic in CI/CD)
|
||||
- **Scripts**: Require `git` and `go` in PATH
|
||||
|
||||
All are standard tools in development and CI/CD environments.
|
||||
|
||||
Created and maintained by Nori.
|
||||
90
cmd/bd/docs.md
Normal file
90
cmd/bd/docs.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# Noridoc: cmd/bd
|
||||
|
||||
Path: @/cmd/bd
|
||||
|
||||
### Overview
|
||||
|
||||
The `cmd/bd` directory contains the complete CLI application for the Beads issue tracker. It implements the `bd` command-line tool, which users interact with to create, query, manage, and synchronize issues across distributed systems.
|
||||
|
||||
The CLI is built on the Cobra framework and consists of command implementations for core operations (create, list, delete, import, export, sync, etc.), daemon management for background operations, and version reporting that includes git commit and branch information from the build.
|
||||
|
||||
### How it fits into the larger codebase
|
||||
|
||||
- **Entry Point**: The CLI defined here (`cmd/bd/main.go`) is the user-facing interface to the entire beads system. All user interactions flow through this package.
|
||||
|
||||
- **Integration with Core Libraries**: The CLI commands call into libraries at `@/internal/beads` (database discovery, version detection), `@/internal/storage` (database operations), `@/internal/rpc` (daemon communication), and other internal packages.
|
||||
|
||||
- **Daemon Communication**: Commands use RPC client logic to communicate with the background daemon (PersistentPreRun hook), allowing the CLI to operate either in daemon mode (delegating to the daemon) or direct mode (local database operations).
|
||||
|
||||
- **Version Reporting**: The version command (`@/cmd/bd/version.go`) reports full build information - it resolves git commit and branch from ldflags set at build time via the Makefile (`@/Makefile`) and goreleaser config (`@/.goreleaser.yml`). This enables users to identify exactly what code their binary was built from.
|
||||
|
||||
- **Release Pipeline Integration**: The version infrastructure here integrates directly with the release pipeline documented in `@/RELEASING.md`. Version information is injected during builds by build automation, ensuring consistency across all distribution channels (GitHub releases, Homebrew, npm, direct go install).
|
||||
|
||||
### Core Implementation
|
||||
|
||||
**Version Information Pipeline** (GitHub issue #503):
|
||||
|
||||
1. **Version Variables** (lines 15-23 in `@/cmd/bd/version.go`):
|
||||
- `Version`: The semantic version (e.g., "0.29.0")
|
||||
- `Build`: The build type ("dev" for local builds, typically set to short commit hash by goreleaser)
|
||||
- `Commit`: Git commit hash (optional, set via ldflag `-X main.Commit=...`)
|
||||
- `Branch`: Git branch name (optional, set via ldflag `-X main.Branch=...`)
|
||||
|
||||
2. **Resolution Strategy** (`resolveCommitHash()` and `resolveBranch()` functions):
|
||||
- First checks if the ldflag was set at build time (most authoritative)
|
||||
- Falls back to `runtime/debug.ReadBuildInfo()` to extract VCS info automatically embedded by Go when using `go build` directly from source
|
||||
- For branch, additionally falls back to `git symbolic-ref --short HEAD` as a runtime resolution
|
||||
|
||||
3. **Build-Time Injection**:
|
||||
- **Makefile** (`@/Makefile`, lines 37-41): The `make install` target extracts git info at build time and passes it to `go install` via ldflags
|
||||
- **.goreleaser.yml** (`@/.goreleaser.yml`): All 5 build configurations (linux-amd64, linux-arm64, darwin-amd64, darwin-arm64, windows-amd64) set the same ldflags using goreleaser's template variables `{{.Commit}}` and `{{.Branch}}`
|
||||
- **scripts/install.sh** (`@/scripts/install.sh`, lines 13-14): Helper script for users who want to build from source with explicit git info passed to `go install`
|
||||
|
||||
4. **Output Formatting**:
|
||||
- **Text Output** (lines 52-58 in `version.go`): Shows format like `bd version 0.29.0 (dev: main@7e70940)` when both commit and branch are available
|
||||
- **JSON Output** (lines 39-50): Includes optional `commit` and `branch` fields when available
|
||||
|
||||
5. **Daemon Version Checking** (lines 63-109): The `--daemon` flag shows daemon/client compatibility by calling health RPC endpoints
|
||||
|
||||
**Command Structure**:
|
||||
- All commands follow the Cobra pattern with `Command` structs and run functions
|
||||
- Commands register themselves via `init()` functions that add them to `rootCmd`
|
||||
- The daemon connection state is managed via PersistentPreRun hooks, allowing most commands to transparently work in daemon or direct mode
|
||||
|
||||
**Key Data Paths**:
|
||||
- User input → Cobra command parsing → Internal beads library calls → Storage layer → Git operations
|
||||
- Responses flow back through storage → RPC (if daemon) or direct return → formatted output
|
||||
|
||||
### Things to Know
|
||||
|
||||
**Why Both Commit and Branch Are Needed**:
|
||||
- The `Commit` variable allows tracing the exact code version
|
||||
- The `Branch` variable provides context for CI/CD systems, build automation, and helps users understand which development line they're running
|
||||
|
||||
**The Problem This Solves** (Issue #503):
|
||||
- `go install` from source doesn't automatically embed VCS info like `go build` does (Go 1.18+ feature)
|
||||
- Without explicit ldflags, users running `bd version` would only see semantic version and build type, not which commit they had
|
||||
- This made it impossible to debug issues or understand build provenance
|
||||
- The fix ensures both `make install` and raw `go install` produce binaries with full version info by setting ldflags explicitly
|
||||
|
||||
**Fallback Resolution Chain** (important for development):
|
||||
- The `resolveCommitHash()` function first checks the ldflag, then checks runtime build info, returning empty if neither is available
|
||||
- This allows the version command to work even in development environments where ldflags aren't set (useful for testing)
|
||||
|
||||
**Testing Coverage** (`@/cmd/bd/version_test.go`):
|
||||
- `TestResolveCommitHash`: Verifies ldflag values are used when set
|
||||
- `TestResolveBranch`: Verifies ldflag values are used when set
|
||||
- `TestVersionOutputWithCommitAndBranch`: Verifies text and JSON output formats correctly include commit and branch information
|
||||
- Existing `TestVersionCommand` and `TestVersionFlag` tests verify basic version output
|
||||
|
||||
**Build System Dependencies**:
|
||||
- The Makefile must have bash and git available at install time
|
||||
- Goreleaser relies on git tags being present for version metadata
|
||||
- Scripts/install.sh is designed for local development from source clones
|
||||
|
||||
**Platform-Specific Considerations**:
|
||||
- The git extraction in Makefile uses POSIX shell constructs compatible with bash on all platforms
|
||||
- Windows builds via goreleaser set ldflags identically to Unix platforms
|
||||
- The `symbolic-ref` fallback in resolveBranch works reliably even in fresh repos with no commits
|
||||
|
||||
Created and maintained by Nori.
|
||||
109
scripts/docs.md
Normal file
109
scripts/docs.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Noridoc: scripts
|
||||
|
||||
Path: @/scripts
|
||||
|
||||
### Overview
|
||||
|
||||
The `scripts` directory contains build and release automation utilities for the beads project. These scripts handle version management, installation with embedded build info, and release orchestration across multiple distribution channels.
|
||||
|
||||
Key scripts include version bumping, installation helpers that inject git information at build time, and release coordination across GitHub, Homebrew, PyPI, and npm.
|
||||
|
||||
### How it fits into the larger codebase
|
||||
|
||||
- **Build Integration**: The `install.sh` script is referenced in documentation and release processes as the primary user-facing installation mechanism. It integrates directly with the Go build system to ensure full version information is embedded.
|
||||
|
||||
- **Version Pipeline**: Works alongside the version infrastructure in `@/cmd/bd/version.go` by extracting and passing git information at build time via ldflags.
|
||||
|
||||
- **Release Automation**: The `release.sh` and `bump-version.sh` scripts orchestrate the release process documented in `@/RELEASING.md`, ensuring version consistency across all components (CLI, plugin, MCP server, npm package).
|
||||
|
||||
- **CI/CD Integration**: Goreleaser (configured in `@/.goreleaser.yml`) uses the same ldflag patterns established by these scripts, ensuring consistency across all installation methods.
|
||||
|
||||
- **Multi-Channel Distribution**: These scripts ensure that whether a user installs via Homebrew, npm, GitHub releases, or direct `go install`, they get consistent version reporting with full git information.
|
||||
|
||||
### Core Implementation
|
||||
|
||||
**install.sh** (GitHub issue #503 fix):
|
||||
|
||||
The script simplifies local installation from source while ensuring full version information is available in the resulting binary:
|
||||
|
||||
1. **Usage** (lines 1-6):
|
||||
- Can be invoked from source checkout with optional custom install directory
|
||||
- `./scripts/install.sh` installs to `$(go env GOPATH)/bin`
|
||||
- `./scripts/install.sh /usr/local/bin` installs to custom location
|
||||
|
||||
2. **Git Information Extraction** (lines 12-14):
|
||||
- Extracts full commit hash via `git rev-parse HEAD`
|
||||
- Extracts branch name via `git rev-parse --abbrev-ref HEAD`
|
||||
- Gracefully handles missing git info (returns empty strings in non-git environments)
|
||||
|
||||
3. **Build Information Display** (lines 16-18):
|
||||
- Shows user where installation will occur
|
||||
- Displays the 12-character short commit hash
|
||||
- Displays the branch name for context
|
||||
|
||||
4. **Installation with Ldflags** (line 20):
|
||||
- Calls `go install` with explicit `-ldflags` to set `main.Commit` and `main.Branch`
|
||||
- These ldflags inject values that are then picked up by `resolveCommitHash()` and `resolveBranch()` in `@/cmd/bd/version.go`
|
||||
|
||||
5. **Post-Install Verification** (lines 22-24):
|
||||
- Immediately runs `bd version` to show the user that installation succeeded
|
||||
- User sees commit and branch info in the output, confirming full version info is present
|
||||
|
||||
**Makefile Integration** (`@/Makefile`, lines 37-41):
|
||||
|
||||
The Makefile's `install` target uses identical logic:
|
||||
- Extracts git info at build time
|
||||
- Passes to `go install` via ldflags
|
||||
- Ensures that standard `make install` produces binaries with full version info, not just `make build`
|
||||
|
||||
**Goreleaser Configuration** (`@/.goreleaser.yml`):
|
||||
|
||||
All 5 platform builds (linux-amd64, linux-arm64, darwin-amd64, darwin-arm64, windows-amd64) use goreleaser's built-in git variables:
|
||||
- `-X main.Commit={{.Commit}}` uses goreleaser's detected commit
|
||||
- `-X main.Branch={{.Branch}}` uses goreleaser's detected branch
|
||||
- Ensures released binaries have full version info without requiring manual extraction
|
||||
|
||||
**Version Bumping** (`bump-version.sh` and `release.sh`):
|
||||
|
||||
- Coordinates version updates across multiple files (CLI version, plugin metadata, MCP server, npm package)
|
||||
- Ensures all distribution channels report consistent version numbers
|
||||
- Integrates with release process documented in `@/RELEASING.md`
|
||||
|
||||
### Things to Know
|
||||
|
||||
**Why Explicit Ldflags Are Necessary**:
|
||||
- `go install` does not automatically embed VCS information like `go build` does (even though Go supports it)
|
||||
- Without explicit ldflags, binaries lack commit and branch information regardless of installation method
|
||||
- The Makefile and goreleaser configurations compensate for this limitation
|
||||
|
||||
**Installation Path Resolution**:
|
||||
- The script uses `$(go env GOPATH)/bin` as the default installation target
|
||||
- This respects the user's Go configuration and matches standard Go tooling behavior
|
||||
- Allows overriding for system-wide installations (e.g., `/usr/local/bin`)
|
||||
|
||||
**Git Information Fallbacks**:
|
||||
- The script silently handles missing git info (returns empty strings)
|
||||
- This allows installation in non-git environments or git-less distributions
|
||||
- The version command in `@/cmd/bd/version.go` has its own fallback chain
|
||||
|
||||
**Testing the Version Pipeline**:
|
||||
- After running `./scripts/install.sh`, users should immediately see full version info via `bd version`
|
||||
- The text output shows format like: `bd version 0.29.0 (dev: main@7e70940)`
|
||||
- JSON output includes both `commit` and `branch` fields
|
||||
|
||||
**Release Coordination**:
|
||||
- The `install.sh` script is independent of release automation
|
||||
- Users can run it locally to build from any source branch
|
||||
- Release scripts (`release.sh`, `update-homebrew.sh`) handle orchestration across channels and are documented separately in `@/RELEASING.md`
|
||||
|
||||
**Platform Compatibility**:
|
||||
- All scripts use POSIX shell constructs (bash on all platforms)
|
||||
- Git operations work identically on macOS, Linux, and Windows (with Git for Windows)
|
||||
- The go install command behaves consistently across all platforms
|
||||
|
||||
**Security Considerations**:
|
||||
- Scripts use `set -e` to fail fast on any errors
|
||||
- Git commands are defensive (using `2>/dev/null` to suppress errors)
|
||||
- No shell injection risks as git values are passed as structured arguments
|
||||
|
||||
Created and maintained by Nori.
|
||||
Reference in New Issue
Block a user