Interest check: to_openeval()/from_openeval() bridge between Dataset/Datapoint and the EvalPort portable format

[adhabnr-ux]

Hi maintainers,

I maintain EvalPort (https://github.com/adhabnr-ux/evalport), an open, provider-neutral JSON Schema spec for portable LLM eval test suites and result sets, meant as a common interchange format so eval data isn't locked to one platform. Spec: https://github.com/adhabnr-ux/evalport/blob/main/SPEC.md

I know from your README that this library is Fern-generated and PRs against the generated code can't land as-is ("we suggest opening an issue first to discuss with us") — so this is exactly that: an interest check before I'd sketch anything, not a PR against generated types.

Looking at src/humanloop/types/dataset_response.py and datapoint_response.py, DatasetResponse.datapoints is a list[DatapointResponse], and each DatapointResponse already carries messages: list[ChatMessage] and a target: dict[str, DatapointResponseTargetValue] — which maps quite directly onto an EvalPort test case:

# sketch, not a PR
def datapoint_to_openeval_testcase(dp: DatapointResponse) -> dict:
    return {
        "id": dp.id,
        "input": [m.content for m in (dp.messages or []) if m.content],
        "expected_output": (dp.target or {}).get("output"),
        "graders": ["exact_match"],
    }

EvalPort test case shape: https://github.com/adhabnr-ux/evalport/blob/main/spec/schemas/testcase.json

Given a Dataset can already be round-tripped through client.prompts.log(...) / client.datasets, a converter like this would let a Humanloop Dataset be exported as a portable, framework-agnostic regression suite (or vice versa). Given the generation constraint, I'd guess this would live as a small hand-written helper module rather than touching the generated types/ — happy to sketch that if there's interest, or to hear if this is better raised against your API/generation pipeline instead. No pressure either way — for context, related precedent is forming in openai/openai-python#3619 (openai/openai-python#3619, still open) where native to_openeval()/from_openeval() helpers are being added to that SDK for the same reason.