feat(parallel_beads): Enhanced validation reporting in PR descriptions
- Added ERROR status for execution failures (command not found, permission error) - Added status definitions: PASS, FAIL, SKIP, ERROR with clear criteria - Restructured PR template with three validation sections: - Automated Checks: table of executed checks with status - Manual Verification Required: unchecked boxes from plan - CONTRIBUTING.md Compliance: extracted requirements with verification status - Added instructions to extract manual verification items from plans - Enhanced CONTRIBUTING.md extraction to track automated vs manual requirements - Updated validation summary format to include error count
This commit was merged in pull request #33.
This commit is contained in:
@@ -135,11 +135,16 @@ Worktree path: [WORKTREE_PATH]
|
||||
- Extract each verification command (lines starting with `- [ ]` followed by a command)
|
||||
- Example: `- [ ] Tests pass: \`make test\`` → extract `make test`
|
||||
- Note any "Per CONTRIBUTING.md:" requirements for additional validation
|
||||
- Also read the "Manual Verification" section from the plan if present
|
||||
- Save manual verification items for inclusion in the PR description (they won't be executed)
|
||||
- If no plan exists, use best-effort validation:
|
||||
- Check if `Makefile` exists → try `make test` and `make lint`
|
||||
- Check if `flake.nix` exists → try `nix flake check`
|
||||
- Check if `package.json` exists → try `npm test`
|
||||
- **Check for CONTRIBUTING.md** → read and extract testing/linting requirements
|
||||
- Track which requirements can be automated vs need manual review
|
||||
- Automated: commands that can be run (e.g., "run `make test`")
|
||||
- Manual: qualitative checks (e.g., "ensure documentation is updated")
|
||||
- If none found, note "No validation criteria found"
|
||||
|
||||
5. **Implement the changes**:
|
||||
@@ -153,8 +158,16 @@ Worktree path: [WORKTREE_PATH]
|
||||
VALIDATION_RESULTS:
|
||||
- make test: PASS
|
||||
- make lint: FAIL (exit code 1: src/foo.ts:23 - missing semicolon)
|
||||
- nix flake check: SKIP (command not found)
|
||||
- nix flake check: SKIP (not applicable - no flake.nix)
|
||||
- cargo test: ERROR (command not found)
|
||||
```
|
||||
|
||||
**Status definitions:**
|
||||
- **PASS**: Check executed successfully with no issues
|
||||
- **FAIL**: Check executed but found issues that need attention
|
||||
- **SKIP**: Check not applicable to this project (e.g., no Makefile for `make test`)
|
||||
- **ERROR**: Check could not execute (missing tool, permission error, command not found)
|
||||
|
||||
- If any validation fails:
|
||||
- Continue with PR creation (don't block)
|
||||
- Document failures in bead notes: `bd update [BEAD_ID] --notes="Validation failures: [list]"`
|
||||
@@ -187,14 +200,27 @@ Worktree path: [WORKTREE_PATH]
|
||||
## Changes
|
||||
- [List of changes made]
|
||||
|
||||
## Validation
|
||||
[Include validation results from step 4]
|
||||
## Validation Steps Completed
|
||||
|
||||
### Automated Checks
|
||||
| Check | Status | Details |
|
||||
|-------|--------|---------|
|
||||
| make test | PASS | |
|
||||
| make lint | FAIL | src/foo.ts:23 - missing semicolon |
|
||||
| nix flake check | SKIP | command not found |
|
||||
| nix flake check | SKIP | not applicable - no flake.nix |
|
||||
| cargo test | ERROR | command not found |
|
||||
|
||||
### Manual Verification Required
|
||||
[If plan has Manual Verification items, list them as unchecked boxes:]
|
||||
- [ ] Verify UI changes match design mockups
|
||||
- [ ] Test on mobile viewport sizes
|
||||
[If no manual verification items: "None specified in plan."]
|
||||
|
||||
### CONTRIBUTING.md Compliance
|
||||
[If CONTRIBUTING.md requirements were extracted:]
|
||||
- [x] Tests pass (verified via `make test`)
|
||||
- [ ] Documentation updated (needs manual review)
|
||||
[If no CONTRIBUTING.md: "No contribution guidelines found."]
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
@@ -214,14 +240,27 @@ Worktree path: [WORKTREE_PATH]
|
||||
## Changes
|
||||
- [List of changes made]
|
||||
|
||||
## Validation
|
||||
[Include validation results from step 4]
|
||||
## Validation Steps Completed
|
||||
|
||||
### Automated Checks
|
||||
| Check | Status | Details |
|
||||
|-------|--------|---------|
|
||||
| make test | PASS | |
|
||||
| make lint | FAIL | src/foo.ts:23 - missing semicolon |
|
||||
| nix flake check | SKIP | command not found |"
|
||||
| nix flake check | SKIP | not applicable - no flake.nix |
|
||||
| cargo test | ERROR | command not found |
|
||||
|
||||
### Manual Verification Required
|
||||
[If plan has Manual Verification items, list them as unchecked boxes:]
|
||||
- [ ] Verify UI changes match design mockups
|
||||
- [ ] Test on mobile viewport sizes
|
||||
[If no manual verification items: None specified in plan.]
|
||||
|
||||
### CONTRIBUTING.md Compliance
|
||||
[If CONTRIBUTING.md requirements were extracted:]
|
||||
- [x] Tests pass (verified via make test)
|
||||
- [ ] Documentation updated (needs manual review)
|
||||
[If no CONTRIBUTING.md: No contribution guidelines found.]"
|
||||
```
|
||||
|
||||
8. **Update bead status**:
|
||||
@@ -235,8 +274,8 @@ PR: [PR_URL]"`
|
||||
- PR URL
|
||||
- Bead ID
|
||||
- Implementation status (success/failure/blocked)
|
||||
- Validation summary: `X passed, Y failed, Z skipped`
|
||||
- List of any validation failures with details
|
||||
- Validation summary: `X passed, Y failed, Z skipped, W errors`
|
||||
- List of any validation failures or errors with details
|
||||
- If blocked or unable to complete, explain what's blocking progress
|
||||
- If validation failed, include the specific failures so the main agent can summarize them for the user
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user