Feature Request: Add Automated Non-Regression Testing to Reduce Bugs and Regressions

[malys]

Summary

The project is evolving quickly and new releases frequently introduce bugs or regressions in previously working features.

Adding a stronger automated non-regression testing strategy would significantly improve overall stability and user confidence.

Motivation

Several regressions have recently been observed in areas such as:

• Remote SSH workspaces
• Delegate functionality
• Authentication
• Localization
• Kanban workflows
• Auto-continue and session recovery
• Configuration handling

Many of these issues appear to affect features that were working correctly in previous versions.

Proposed Solution

Introduce automated non-regression tests as part of the development and release process.

Suggested Coverage

SSH Remote Workspaces

Test:

• Workspace creation
• Remote command execution
• Delegate on remote workspace
• File browser access
• Reconnect after disconnect
• Path handling (Linux vs Windows)

Kanban

Test:

• Task creation
• Task updates
• Task completion
• Task status transitions
• Task recovery after failure

Agent Execution

Test:

• Max turns reached
• Auto-continue behavior
• Restart Session
• Session recovery
• Usage limit handling

Authentication

Test:

• OAuth login
• OAuth refresh
• Expired session recovery
• Logout/login cycle

Localization

Test:

• English locale
• Translation fallback
• Missing translation keys

Expected result:

Missing translation
→ English fallback

Never:

English UI
→ Ukrainian text

Configuration

Test:

• Global settings
• Workspace settings
• Settings precedence
• Persistence across restarts

CI/CD Integration

Run tests automatically:

Pull Request
      ↓
Build
      ↓
Regression Tests
      ↓
Merge Allowed

Release builds should only be published when critical regression suites pass.

Additional Suggestions

Regression Test Suite

Maintain a dedicated suite:

Regression Tests

containing reproductions of previously reported bugs.

Example:

Issue #67

Once fixed:

Regression Test #67

to ensure it never reappears.

Benefits

• Fewer regressions.
• More reliable releases.
• Faster validation of fixes.
• Better confidence when refactoring.
• Reduced support burden.
• Improved user experience.

Priority

High

Use Case

As a user, I want critical workflows to be covered by automated non-regression tests so that upgrades do not reintroduce bugs in features that were previously working.