Skip to content

LLM interpretation: --interpret flag for failure explanation #18

Description

@metasmile

Goal

Implement the --interpret flag for ev verify that passes failure results through an LLM for natural-language explanation of why each encoding failed.

Background

The --interpret flag is documented in CLI examples but not implemented. It should run a lightweight LLM pass over failures and explain why each one occurred.

Design considerations

  • API key management via environment variable (EV_LLM_API_KEY)
  • Prompt template design: provide field spec, constraint spec, and the specific failing combination
  • Rate limiting and caching of explanations (same failure pattern → same explanation)
  • Optional: support multiple LLM backends (OpenAI, Anthropic, local via Ollama)

Deliverables

  1. --interpret flag on ev verify and ev simulate subcommands
  2. LLM client module with configurable backend
  3. Prompt template for constraint violation explanation
  4. Rate limiting and result caching
  5. Integration with existing TextReporter (append explanations to text output)

References

  • src/main.rs — CLI subcommands
  • src/reporter.rs — ReporterCapable trait
  • Cargo.toml — already has reqwest dependency (for HTTP calls)

Parent Issue

#13 — Phase 2: Spike simulation backend and CVA6 XIF exhaustive verification

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions