Testing: Replace ambiguous CLI exit-code assertions

[bcherrington]

Current gap

The CLI suite still contains many broad assertions such as membership in multiple exit codes or generic non-zero checks. These assertions can allow a command to fail for the wrong reason.

Acceptance criteria

• Inventory broad exit-code assertions and classify expected success, usage error, domain failure, cancellation, and unsupported behavior.
• Replace ambiguous assertions with exact expectations plus relevant output or exception context.
• Reuse shared diagnostics from Testing: Finish adoption of shared CLI test utilities #54 without obscuring intent.
• Align expectations with the behavior contract in CLI: Standardize actionable errors and exit-code contracts #7.
• Keep deliberate multi-platform differences explicit and narrowly justified.
• Run the complete CLI suite and normal CI profile.

[github-actions]

🔍 Issue Triage Analysis

Priority Level: Medium-High

This issue addresses test reliability and debugging efficiency, which are critical for maintaining code quality and developer productivity. While not blocking core functionality, it significantly impacts the development workflow.

Issue Type: Enhancement (Testing Infrastructure)

This is a testing infrastructure improvement that enhances the precision and maintainability of the CLI test suite.

Complexity Estimate: Medium

• Scope: 9 test files need systematic updates
• Technical Complexity: Moderate - requires understanding of exit code semantics and test patterns
• Implementation: Well-defined approach with clear examples and standards
• Risk: Low - changes are isolated to test assertions with clear before/after patterns

Suggested Labels Applied:

• priority:medium-high - Important for development workflow
• area:testing - Testing infrastructure improvement
• complexity:medium - Moderate scope and technical requirements
• enhancement - Improvement to existing functionality
• cli - CLI-related work
• error-handling - Improves error validation patterns

Team/Component: Testing & CLI

This issue spans testing infrastructure and CLI validation, requiring familiarity with:

• Typer CLI framework and exit code conventions
• Pytest testing patterns
• CLI error handling semantics

Dependencies Analysis:

✅ Clear Dependency: Issue #54 (Apply shared test utilities) - Must be completed first

• The issue correctly identifies this dependency
• Shared utilities from Testing: Finish adoption of shared CLI test utilities #54 provide the foundation for standardized assertions
• Implementation should wait until Testing: Finish adoption of shared CLI test utilities #54 is resolved

Reproducibility: Excellent

• Clear examples of current problematic patterns
• Specific target patterns with code examples
• Well-defined scope with file-by-file checklist
• Concrete acceptance criteria

Acceptance Criteria: Well-Defined

The issue provides comprehensive acceptance criteria:

• ✅ Specific assertion patterns to replace
• ✅ Clear exit code standards documented
• ✅ Implementation strategy outlined
• ✅ Testing approach defined

Similar Issues:

Builds upon #8 (CLI test suite implementation) and directly depends on #54. Part of the broader testing infrastructure improvement initiative.

📋 Recommendations

Next Steps:

1. Wait for Testing: Finish adoption of shared CLI test utilities #54 completion - This issue cannot proceed until shared test utilities are available
2. Start with error handling tests as suggested - test_cli_error_handling.py has the most complex scenarios
3. Process files incrementally - Update and test each file individually to prevent regressions
4. Validate exit code standards - Ensure the documented exit codes align with Typer framework behavior

Implementation Approach:

1. Audit Phase: Systematically identify all broad exit code assertions across the 9 target files
2. Categorization: Group assertions by expected behavior (success, command error, application error)
3. Incremental Updates: Update files one at a time with thorough testing
4. Validation: Ensure all existing tests continue to pass with improved diagnostics

Potential Assignees:

Developers with experience in:

• CLI testing patterns and Typer framework
• Pytest and test infrastructure
• Error handling and exit code semantics

Immediate Actions:

• ✅ Labels applied and priority set
• ⏳ Block until Testing: Finish adoption of shared CLI test utilities #54 is complete
• 📝 Consider creating a tracking checklist for the 9 files to monitor progress

Risk Assessment: Low

• Changes are isolated to test assertions
• Clear patterns for before/after transformations
• Existing functionality remains unchanged
• Comprehensive acceptance criteria reduce implementation risk

---

Issue Type Recommendation: Enhancement

This is a well-structured enhancement that will significantly improve the testing infrastructure's precision and maintainability. The clear dependency on #54 and systematic approach make it ready for implementation once the prerequisite is complete.

[bcherrington]

Reconciled on 2026-07-18. Retained as current test debt: repository search found 123 broad exit-code or shared-utility-related matches, including explicit multi-code assertions.