Skip to content

enhancement: add provenance and confidence boundaries to evidence-backed log explanations #21

Description

@CoreyLeath-code

Audit finding

LogSight-AI already has project-specific reasoning work in logsight/cli.py and tests/test_reasoning.py. A repository search found no confidence_score implementation, so explanation certainty and evidence provenance are not yet a stable output contract.

Evidence inspected

  • README.md: local-first CLI; it parses logs, finds outliers/spikes, and does not claim learned incident-detection accuracy.

  • logsight/cli.py and tests/test_reasoning.py: reasoning-related implementation and tests.

  • pyproject.toml, CI, and test files are present.

Proposed enhancement

Extend the existing explanation result, not the deployment model, with deterministic event references, analyzer/rule identifiers, support-based confidence labels, and an explicit insufficient-evidence outcome. Keep logs local and do not infer certainty from an LLM.

Acceptance criteria

  • Preserve local-only operation and backwards-compatible CLI output, or version the output.

  • Add tests for corroborated, conflicting, empty, malformed, and boundary-size inputs.

  • Describe the deterministic confidence basis and redaction limitations.

  • State that results are diagnostic assistance, not an incident verdict.

NEEDS HUMAN DECISION

Choose whether provenance can expose raw log lines, hashes, or line ranges; raw lines may contain sensitive data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions