From 2618f9fed298c2a522ffffd581f2197600d5742b Mon Sep 17 00:00:00 2001 From: Steve Yegge Date: Sat, 8 Nov 2025 03:53:08 -0800 Subject: [PATCH] Prepare v0.23.0 release: update CHANGELOG and remove artifacts --- AGENT_MAIL_INTEGRATION_STATUS.md | 53 ------- CACHE_AUDIT.md | 238 ------------------------------- CHANGELOG.md | 83 +++++++++++ git_traffic_report.md | 49 ------- latency_benchmark_results.md | 6 - latency_results.md | 54 ------- 6 files changed, 83 insertions(+), 400 deletions(-) delete mode 100644 AGENT_MAIL_INTEGRATION_STATUS.md delete mode 100644 CACHE_AUDIT.md delete mode 100644 git_traffic_report.md delete mode 100644 latency_benchmark_results.md delete mode 100644 latency_results.md diff --git a/AGENT_MAIL_INTEGRATION_STATUS.md b/AGENT_MAIL_INTEGRATION_STATUS.md deleted file mode 100644 index deca3406..00000000 --- a/AGENT_MAIL_INTEGRATION_STATUS.md +++ /dev/null @@ -1,53 +0,0 @@ -# MCP Agent Mail Integration - Current Status - -## Proof of Concept ✅ COMPLETE - -**Epic:** bd-spmx (Investigation & Proof of Concept) - CLOSED - -### Completed Validation -- ✅ **bd-muls**: Server installed and tested (~/src/mcp_agent_mail) -- ✅ **bd-27xm**: MCP API tool execution issues resolved -- ✅ **bd-6hji**: File reservation collision prevention validated - - Two agents (BrownBear, ChartreuseHill) tested - - First agent gets reservation, second gets conflict - - Collision prevention works as expected -- ✅ **bd-htfk**: Latency benchmarking completed - - Agent Mail: <100ms (HTTP API round-trip) - - Git sync: 2000-5000ms (full cycle) - - **20-50x latency reduction confirmed** -- ✅ **bd-pmuu**: Architecture Decision Record created - - File: [docs/adr/002-agent-mail-integration.md](docs/adr/002-agent-mail-integration.md) - - Documents integration approach, alternatives, tradeoffs - -### Validated Benefits -1. **Collision Prevention**: Exclusive file reservations prevent duplicate work -2. **Low Latency**: <100ms vs 2000-5000ms (20-50x improvement) -3. **Lightweight**: <50MB memory, simple HTTP API -4. **Optional**: Git-only mode remains fully supported - -## Next Phase: Integration (bd-wfmw) - -Ready to proceed with integration layer implementation: -- HTTP client wrapper for Agent Mail API -- Reservation checks in bd update/ready -- Graceful fallback when server unavailable -- Environment-based configuration - -## Quick Start Commands -```bash -# Start Agent Mail server (optional) -cd ~/src/mcp_agent_mail -source .venv/bin/activate -uv run python -m mcp_agent_mail.cli serve-http - -# Access web UI -open http://127.0.0.1:8765/mail - -# Stop server -pkill -f "mcp_agent_mail.cli" -``` - -## Resources -- [Latency Benchmark Results](latency_results.md) -- [ADR 002: Agent Mail Integration](docs/adr/002-agent-mail-integration.md) -- [Agent Mail Repository](https://github.com/Dicklesworthstone/mcp_agent_mail) diff --git a/CACHE_AUDIT.md b/CACHE_AUDIT.md deleted file mode 100644 index 94bab19e..00000000 --- a/CACHE_AUDIT.md +++ /dev/null @@ -1,238 +0,0 @@ -# Cache Removal Audit - Complete ✅ - -**Issue:** [bd-bc2c6191](file:///Users/stevey/src/dave/beads/.beads) - Audit Current Cache Usage -**Date:** 2025-11-06 -**Status:** Cache has already been removed successfully - -## Executive Summary - -**The daemon storage cache has already been completely removed** in commit `322ab63` (2025-10-28). This audit confirms: - -✅ Cache implementation deleted -✅ No references to cache remain in codebase -✅ MCP multi-repo support works correctly without cache -✅ All environment variables removed -✅ All tests updated and passing - -## Investigation Results - -### 1. Cache Implementation Status - -**File:** `internal/rpc/server_cache_storage.go` -**Status:** ❌ DELETED in commit `322ab63` - -**Evidence:** -```bash -$ git show 322ab63 --stat - internal/rpc/server_cache_storage.go | 286 ----------- - internal/rpc/server_eviction_test.go | 525 --------------------- - 10 files changed, 6 insertions(+), 964 deletions(-) -``` - -**Removed code:** -- `server_cache_storage.go` (~286 lines) - Cache implementation -- `server_eviction_test.go` (~525 lines) - Cache eviction tests -- Cache fields from `Server` struct -- Cache metrics from health/metrics endpoints - -### 2. Client Request Routing - -**File:** `internal/rpc/client.go` -**Status:** ✅ SIMPLIFIED - No cache references - -**Key findings:** -- `req.Cwd` is set in `ExecuteWithCwd()` (line 108-124) -- Used for database discovery, NOT for multi-repo routing -- Falls back to `os.Getwd()` if not provided -- Sent to daemon for validation only - -**Code:** -```go -// ExecuteWithCwd sends an RPC request with an explicit cwd (or current dir if empty string) -func (c *Client) ExecuteWithCwd(operation string, args interface{}, cwd string) (*Response, error) { - // Use provided cwd, or get current working directory for database routing - if cwd == "" { - cwd, _ = os.Getwd() - } - - req := Request{ - Operation: operation, - Args: argsJSON, - ClientVersion: ClientVersion, - Cwd: cwd, // For database discovery - ExpectedDB: c.dbPath, // For validation - } - // ... -} -``` - -### 3. Server Storage Access - -**Status:** ✅ SIMPLIFIED - Direct storage access - -**Previous (with cache):** -```go -store := s.getStorageForRequest(req) // Dynamic routing via cache -``` - -**Current (without cache):** -```go -store := s.storage // Direct access to local daemon's storage -``` - -**Evidence:** -```bash -$ git show 322ab63 | grep -A2 -B2 "getStorageForRequest" -- store := s.getStorageForRequest(req) -+ store := s.storage -``` - -**Files using `s.storage` directly:** -- `server_issues_epics.go` - All issue CRUD operations -- `server_labels_deps_comments.go` - Labels, dependencies, comments -- `server_routing_validation_diagnostics.go` - Health, metrics, validation -- `server_export_import_auto.go` - Export, import, auto-import -- `server_compact.go` - Compaction operations - -### 4. Environment Variables - -**Status:** ✅ ALL REMOVED - -Searched for: -- `BEADS_DAEMON_MAX_CACHE_SIZE` - ❌ Not found -- `BEADS_DAEMON_CACHE_TTL` - ❌ Not found -- `BEADS_DAEMON_MEMORY_THRESHOLD_MB` - ❌ Not found - -**Remaining daemon env vars (unrelated to cache):** -- `BEADS_DAEMON_MAX_CONNS` - Connection limiting -- `BEADS_DAEMON_REQUEST_TIMEOUT` - Request timeout -- `BEADS_MUTATION_BUFFER` - Event-driven sync buffer - -### 5. MCP Multi-Repo Support - -**Status:** ✅ WORKING WITHOUT CACHE - -**Architecture:** LSP-style per-project daemons (v0.16.0+) - -``` -MCP Server (one instance) - ↓ -Per-Project Daemons (one per workspace) - ↓ -SQLite Databases (complete isolation) -``` - -**How multi-repo works now:** -1. MCP server maintains connection pool keyed by workspace path -2. Each workspace has its own daemon socket (`.beads/bd.sock`) -3. Daemon serves only its local database (`s.storage`) -4. No caching needed - routing happens at connection level - -**From MCP README:** -> The MCP server maintains a connection pool keyed by canonical workspace path: -> - Each workspace gets its own daemon socket connection -> - Paths are canonicalized (symlinks resolved, git toplevel detected) -> - No LRU eviction (keeps all connections open for session) - -**Key files:** -- `integrations/beads-mcp/src/beads_mcp/server.py` - Connection pool management -- `integrations/beads-mcp/src/beads_mcp/tools.py` - Per-request workspace routing via ContextVar -- `integrations/beads-mcp/src/beads_mcp/bd_daemon_client.py` - Daemon client with socket pooling - -### 6. Test Coverage - -**Status:** ✅ ALL TESTS UPDATED - -**Removed tests:** -- `internal/rpc/server_eviction_test.go` (525 lines) - Cache eviction tests -- Cache assertions from `internal/rpc/limits_test.go` (55 lines) - -**Remaining multi-repo tests:** -- `integrations/beads-mcp/tests/test_multi_project_switching.py` - Path canonicalization (LRU cache for path resolution, NOT storage cache) -- `integrations/beads-mcp/tests/test_daemon_health_check.py` - Client connection pooling -- No Go tests reference `getStorageForRequest` or storage cache - -**Evidence:** -```bash -$ grep -r "getStorageForRequest\|cache.*storage" internal/rpc/*_test.go cmd/bd/*_test.go -# No results -``` - -### 7. Stale References - -**File:** `internal/rpc/server.go` -**Status:** ⚠️ STALE COMMENT - -**Line 6:** -```go -// - server_cache_storage.go: Storage caching, eviction, and memory pressure management -``` - -**Action needed:** Remove this line from comment block - -## Architecture Change Summary - -### Before (with cache) -``` -Client Request - ↓ -req.Cwd → getStorageForRequest(req) - ↓ -Cache lookup by workspace path - ↓ -Return cached storage OR create new -``` - -### After (without cache) -``` -Client Request - ↓ -Daemon validates req.ExpectedDB == s.storage.Path() - ↓ -Direct access: s.storage - ↓ -Single storage per daemon (one daemon per workspace) -``` - -### Why this works better - -**Problems with cache:** -1. Complex eviction logic (memory pressure, LRU) -2. Risk of cross-workspace data leakage -3. Global daemon serving multiple databases was confusing -4. Cache staleness issues - -**Benefits of per-workspace daemons:** -1. ✅ Complete isolation - one daemon = one database -2. ✅ Simpler mental model -3. ✅ No cache eviction complexity -4. ✅ Follows LSP (Language Server Protocol) pattern -5. ✅ MCP connection pooling handles multi-repo at client level - -## Conclusion - -✅ **Cache removal is complete and successful** - -**No action needed** except: -1. Update stale comment in `internal/rpc/server.go:6` -2. Close this issue (bd-bc2c6191) - -**MCP multi-repo support confirmed working** via: -- Per-project daemon architecture -- Connection pooling at MCP server level -- Path canonicalization with LRU cache (for paths, not storage) - -## Related Issues - -- [bd-bc2c6191] - This audit (ready to close) -- Commit `322ab63` - Cache removal (2025-10-28) -- Commit `9edcb6f` - Remove cache fields from Server struct -- Commit `bbb1725` - Replace getStorageForRequest with s.storage -- Commit `c3786e3` - Add cache usage audit documentation - -## Recommendations - -1. ✅ Close bd-bc2c6191 - Audit complete, cache confirmed removed -2. 🔧 Fix stale comment in `internal/rpc/server.go:6` -3. 📚 Document per-daemon architecture in AGENTS.md (may already exist) -4. ✅ No tests need updating - all passing after cache removal diff --git a/CHANGELOG.md b/CHANGELOG.md index 4eae5ebe..28f09894 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,89 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.23.0] - 2025-11-08 + +### Added + +- **Agent Mail Integration**: Complete Python adapter library with comprehensive documentation and multi-agent coordination tests + - Python adapter library in `integrations/agent-mail-python/` + - Agent Mail quickstart guide and comprehensive integration docs + - Multi-agent race condition tests and failure scenario tests + - Automated git traffic benchmark + - Bash-agent integration example + +- **bd info --whats-new** (bd-eiz9): Agent version awareness for quick upgrade summaries + - Shows last 3 versions with workflow-impacting changes + - Supports `--json` flag for machine-readable output + - Helps agents understand what changed without re-reading full docs + +- **bd hooks install** (bd-908z): Embedded git hooks command + - Replaces external install script with native command + - Git hooks now embedded in bd binary + - Works for all bd users, not just source repo users + +- **bd cleanup**: Bulk deletion command for closed issues (bd-buol) + - Agent-driven compaction for large databases + - Removes closed issues older than specified threshold + +### Fixed + +- **3-way JSONL Merge** (bd-jjua): Auto-invoked on conflicts + - Automatically triggers intelligent merge on JSONL conflicts + - No manual intervention required + - Warning message added to zombie issues.jsonl file + +- **Auto-import on Missing Database** (ab4ec90): `bd import` now auto-initializes database when missing +- **Daemon Crash Recovery** (bd-vcg5): Panic handler with socket cleanup prevents orphaned processes +- **Stale Database Exports** (bd-srwk): ID-based staleness detection prevents exporting stale data +- **Windows MCP Subprocess Timeout** (bd-r79z): Fix for git detection on Windows +- **Daemon Orphaning** (a6c9579): Track parent PID and exit when parent dies +- **Test Pollution Prevention** (bd-z528, bd-2c5a): Safeguards to prevent test issues in production database +- **Client Self-Heal** (a236558): Auto-recovery for stale daemon.pid files +- **Post-Merge Hook Error Messages** (abb1d1c): Show actual error messages instead of silent failures +- **Auto-import During Delete** (bd-8kde): Disable auto-import during delete operations to prevent conflicts +- **MCP Workspace Context** (bd-8zf2): Auto-detect workspace from CWD +- **Import Sync Warning** (bd-u4f5): Warn when import syncs with working tree but not git HEAD +- **GH#254** (bd-tuqd): `bd init` now detects and chains with existing git hooks +- **GH#249**: Add nil storage checks to prevent RPC daemon crashes +- **GH#252**: Fix SQLite driver name mismatch causing "unknown driver" errors +- **Nested .beads Directories** (bd-eqjc): Prevent creating nested .beads directories +- **Windows SQLite Support**: Fix SQLite in releases for Windows + +### Changed + +- **Agent Affordances** (observations from agents using beads): + - **bd new**: Added as alias for `bd create` command (agents often tried this) + - **bd list**: Changed default to one-line-per-issue format to prevent agent miscounting; added `--long` flag for previous detailed format + +- **Developer Experience**: + - Extracted supplemental docs from AGENTS.md for better organization + - Added warning for working tree vs git HEAD sync mismatches + - Completion commands now work without database + - Config included in `bd info` JSON output + - Python cache files added to .gitignore + - RPC diagnostics available via `BD_RPC_DEBUG` env var + - Reduced RPC dial timeout from 2s to 200ms for fast-fail (bd-expt) + - Standardized daemon detection with tryDaemonLock probe (bd-wgu4) + - Improved internal/daemon test coverage to 60% + +- **Code Organization**: + - Refactored snapshot management into dedicated module (bd-urob) + - Documented external_ref in content hash behavior (bd-9f4a) + - Added MCP server functions for repair commands (bd-7bbc4e6a) + - Added version number to beads-mcp startup log + - Added system requirements section for glibc compatibility in docs + +- **Release Automation**: + - Automatic Homebrew formula update in release workflow + - Gitignore Formula/bd.rb (auto-generated, real source is homebrew-beads tap) + +- **Other**: + - Added `docs/` directory to links (#242) + - RPC monitoring solution with web UI as implementation example (#244) + - Remove old install.sh script, replaced by `bd hooks install` + - Remove vc.db exclusion from FindDatabasePath filter + ## [0.22.1] - 2025-11-06 ### Added diff --git a/git_traffic_report.md b/git_traffic_report.md deleted file mode 100644 index 8ad03e72..00000000 --- a/git_traffic_report.md +++ /dev/null @@ -1,49 +0,0 @@ -# Git Traffic Reduction Benchmark - -**Date:** 2025-11-08T02:06:36.626017 -**Issues Processed:** 10 - -## Results - -### Without Agent Mail (Git-only mode) -- **Pulls:** 40 -- **Commits:** 0 -- **Pushes:** 0 -- **Total Git Operations:** 40 - -### With Agent Mail -- **Pulls:** 1 -- **Commits:** 1 -- **Pushes:** 1 -- **Total Git Operations:** 3 - -## Traffic Reduction - -- **Absolute Reduction:** 37 operations -- **Percentage Reduction:** 92.5% -- **Target Reduction:** 70% -- **Status:** ✅ PASS - -## Analysis - -In git-only mode, each issue requires multiple git operations for coordination: -- Pull before checking status -- Commit after status update -- Push to share with other agents -- Pull by other agents to get updates - -With Agent Mail, coordination happens over HTTP: -- No pulls for status checks (Agent Mail inbox) -- No commits for reservations (in-memory) -- Batched commits at strategic sync points -- Single push at end of workflow - -**Expected workflow for 10 issues:** - -| Mode | Operations per Issue | Total Operations | -|------|---------------------|------------------| -| Git-only | ~9 (3 pulls + 3 commits + 3 pushes) | 40 | -| Agent Mail | Batched | 3 | - -**Reduction:** 92.5% fewer git operations - diff --git a/latency_benchmark_results.md b/latency_benchmark_results.md deleted file mode 100644 index a3863310..00000000 --- a/latency_benchmark_results.md +++ /dev/null @@ -1,6 +0,0 @@ -# Latency Benchmark: Agent Mail vs Git Sync - -Date: Sat Nov 8 00:04:08 PST 2025 - -## Git Sync Latency (bd update → commit → push → pull → import) - diff --git a/latency_results.md b/latency_results.md deleted file mode 100644 index 9fe3c1e1..00000000 --- a/latency_results.md +++ /dev/null @@ -1,54 +0,0 @@ -# Agent Mail vs Git Sync Latency Benchmark - -**Test Date:** 2025-11-08 -**Issue:** bd-htfk (Measure notification latency vs git sync) - -## Methodology - -### Git Sync Latency -Measures time for: `create` → `update` → `flush to JSONL` - -This represents the minimum local latency without network I/O. Full git sync (commit + push + pull) would add network RTT (~1000-5000ms). - -### Agent Mail Latency -Server not currently running. Based on previous testing and HTTP API structure, expected latency is <100ms for: `send_message` → `fetch_inbox`. - -## Results - -### Git Sync (Local Flush Only) - -| Run | Latency | -|-----|---------| -| Manual Test 1 | ~500ms | -| Manual Test 2 | ~480ms | -| Manual Test 3 | ~510ms | - -**Average:** ~500ms (local export only, no network) - -With network (commit + push + pull + import): -- **Estimated P50:** 2000-3000ms -- **Estimated P95:** 4000-5000ms -- **Estimated P99:** 5000-8000ms - -### Agent Mail (HTTP API) - -Based on bd-6hji testing and HTTP API design: -- **Measured:** <100ms for send + fetch round-trip -- **P50:** ~50ms -- **P95:** ~80ms -- **P99:** ~100ms - -## Conclusion - -✅ **Agent Mail delivers 20-50x lower latency** than git sync: -- Agent Mail: <100ms (verified in bd-6hji) -- Git sync: 2000-5000ms (estimated for full cycle) - -The latency reduction validates one of Agent Mail's core benefits for real-time agent coordination. - -## Next Steps - -- ✅ Latency advantage confirmed -- ✅ File reservation collision prevention validated (bd-6hji) -- 🔲 Measure git operation reduction (bd-nemp) -- 🔲 Create ADR documenting integration decision (bd-pmuu)