Skip to content

P0 Epic: Generate reviewable TestCaseAssets through testing-design AI generation #792

Description

@wanghuan-520

Summary

Make AI-assisted TestCase generation a first-class V2 testing-design capability. Testing Packages must generate reviewable candidate TestCases from immutable approved analysis inputs, while PQL remains the only authority that approves and publishes TestCaseAssets. Runtime execution remains deterministic and never calls a model to invent or modify actions.

Target branch: dev.
Baseline for preflight: dev@3c7bb47e891010f9254e55eb0b9f3485a0a69fc6 or a then-current descendant.

Product outcome

PQL ProjectPackSnapshot / requirements / journeys / risks
  -> testing-design static analysis
  -> repository-analysis + requirements-index + traceability-seed
  -> TestCaseGenerationPort
  -> Codex CLI adapter
  -> candidate TestCase / Step / typed Action / Assertion
  -> Testing Packages validation, deduplication and review
  -> candidate TestCase artifacts + GenerationReceipt
  -> PQL approval / rejection / publication as TestCaseAsset
  -> PQL TestSelection
  -> deterministic StructuredPlan Compiler and Runner

Ownership

Testing Packages owns:

  • generation request/result contracts;
  • candidate TestCase/Step/Action/Assertion semantics;
  • provider-neutral generation port;
  • Codex CLI adapter;
  • model/prompt/policy provenance;
  • validation, allowlist enforcement, deduplication and review artifacts;
  • deterministic artifact identity and replay/conflict behavior.

PQL owns:

  • ProjectPackSnapshot, requirement/journey/risk context;
  • approval and publication of TestCaseAsset;
  • asset lifecycle and eligibility;
  • TestSelection.

Local QA Runtime and Testing Packages Runner may only execute an approved immutable StructuredPlan. They do not call AI or modify TestCases during execution.

Child issues

  1. P0: Publish testing-design candidate TestCase generation contracts #793 — publish generation contracts, schemas, fixtures and catalog bindings.
  2. P0: Add TestCaseGenerationPort with a Codex CLI adapter #794 — add provider-neutral TestCaseGenerationPort with Codex CLI adapter.
  3. P0: Validate, deduplicate, review, and hand candidate TestCases to PQL #795 — validate, deduplicate, review and hand candidates to PQL approval.

Execution policy:

Dependency graph

#793 contracts
  -> #794 Codex adapter --------┐
  -> #795 validation foundation ├-> #795 final PQL handoff
                                └-> one local Heca repository candidate-generation canary

Child 3 may develop pure validation and deduplication in parallel with Child 2 after Child 1 freezes the contract, but final end-to-end closure requires both.

Epic acceptance criteria

  • A pinned Heca repository revision and approved static-analysis artifacts produce a bounded candidate TestCase set through Codex CLI.
  • Every candidate uses only registered typed actions and assertions.
  • Generation provenance binds exact repository commit, analysis artifacts, prompt template, adapter, model, policy, input digest, raw response digest and validated candidate digest.
  • Same immutable input and generation identity replays the same accepted artifact; conflicts fail closed.
  • Malformed output, refusal, timeout, cancellation, over-budget, unsupported action/assertion, duplicate case and untraceable requirement are bounded explicit outcomes.
  • AI output never becomes an approved TestCaseAsset without the PQL approval/publication boundary.
  • Runtime execution contains no TestCase generation or model call.
  • Focused tests, full tests, checks and provider-secret leakage scans pass.

Out of scope

  • PQL implementation.
  • TestSelection and StructuredPlan Compiler changes except consuming approved TestCaseAssets.
  • Local QA Runtime, Talos, NyxID or Browser execution.
  • Runtime AI action generation or model-judged assertions.
  • Automatic TestCaseAsset promotion or release decisions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions