confidential computing refactoring and features - #706
Draft
hasan7n wants to merge 40 commits into
Draft
Conversation
Everything confidential computing lived under `cli/medperf/asset_management/`, reachable only through the MedPerf client. None of it was reusable: a key broker, an auditor, or anyone writing their own benchmark script had no way to depend on the parts without depending on the whole client. The components now live in `cc/` as `medperf-cc`, a separate installable package. This commit is a move: the classes, their names and their responsibilities are unchanged, and reorganizing them is left to the next one. Two dependencies had to be cut for the package to stand alone, and both cut in the same direction -- key material stays with whoever owns it: - An asset arrives at `AssetStorageManager` already encrypted. The encryption key is the asset owner's, so there was never a reason for it to travel into the storage layer. - Results leave `OperatorManager` still encrypted. Only the holder of the result collector's private key can open them, and that is the client. Progress reporting moved out for the same reason: `medperf_cc` no longer reaches for `config.ui`, so the caller sets the messages it wants shown. What stays in the client is translation, now collected under `medperf/cc/` beside the commands that use it: entity configuration into managers, benchmark associations into permitted workload identities, and MedPerf's own encryption and storage conventions into what the components are handed. Also drops `__gcp_util_archive.py`, 391 lines nothing imported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ckend
The moved package still had every concept tangled with the one cloud that
happens to implement it. `CCWorkloadID` sat in `gcp/types.py` next to
`GCPAssetConfig`; how an owner scopes a grant existed only as a CEL expression
built inside a manager; and the split between an "asset policy manager" and an
"asset storage manager" tracked no boundary -- both took the same config and
both wrote to the same bucket.
Split along what an asset owner actually decides, and what a backend does
about it:
identity what a workload is, and which terms each kind of owner pins
policy where a workload must run for the key to be released
workload the environment contract a confidential workload reads
vault where an asset's ciphertext lives, and who may have its key
operator starting a confidential workload and fetching its output
gcp/ KMS, IAM, GCS and Confidential Space, and nothing else
The terms of an identity are now named once, as attestation claim paths, which
is what every backend reads one way or another. Rendering them as CEL is
Google's business and stays in `gcp/vault.py`; a test pins the two views of a
binding against each other, because if they ever disagreed nothing would
error -- every authorization would silently stop matching.
Collapsing duplicate identities also moves into the vault. It is not the
caller's business to know that a model owner's grant makes two workloads
differing only in dataset the same grant; that follows from the binding, which
the vault holds.
`AssetPolicy` becomes a validated model rather than a free-form dict, and
rejects keys it does not recognize: a policy decides who may read an asset, so
a misspelled key is a policy the owner did not get.
Names lose the `CC` prefix, which said nothing inside a package called
`medperf_cc`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which parties an asset owner pinned was decided by which kind of owner they
were: a data owner authorized one exact (script, data, model, collector)
combination, a model owner authorized a (script, model) pair and nothing else.
Neither could say otherwise, and the two rules lived in separate branches of
separate functions.
Both sides now answer the same two questions in their policy:
bind_peer_asset also pin the other asset in the execution
allowed_result_collectors whose keys results may be encrypted for
Two terms are deliberately not configurable. The benchmark script is always
pinned, or any container image could ask for the key; the owner's own asset is
always pinned, or a workload aimed at somebody else's asset could use this
grant to read theirs.
Naming a collector is what pins the collector. Keeping "which key" separate
from "whether to check it" would allow two states that cannot mean anything: a
pinned key nobody named, which no workload can ever match, and a restriction
with no key to enforce it against, which is advice rather than a control.
An unset choice is not "off". It resolves to the default for the kind of
asset -- data owners pin everything, model owners pin neither -- which is what
each side did before this was a choice, and is also the right direction for
each: data cannot be un-leaked, and a model owner's grant is meant to survive a
new dataset joining a benchmark.
Whichever terms an owner does not pin, they stop having to enumerate: a model
owner who does not pin the dataset produces one grant instead of one per
approved dataset, and the vault collapses the rest.
Result collectors other than the data owner are refused for now. Every
execution encrypts its results for the data owner, so no other party's key
could appear in an attestation, and a policy naming one could never be
satisfied. Opening that up is the next commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Results were always encrypted for the data owner's key, whoever ran the execution. A non-data-owner operator could therefore start a workload and then fail to open what came back, since they held the wrong private key -- so in practice only a data owner could run anything, and `allowed_result_collectors` had nowhere to point but at them. Results are now encrypted for the operator, which is who decrypts them. That makes every party's key a possible collector, and makes an asset owner's list of collectors mean what it says: these are the roles I will let operate an execution involving my asset. Enforced twice, deliberately. The cloud enforces it by refusing to release a key to a workload whose attested collector hash is not one the owner permitted -- but a VM has started by then, and the operator only sees a workload that produced nothing. So the client checks the same thing up front, against both owners' lists, and says which owner refused. An `inference_script` benchmark is the one topology that still requires the data owner, and now says so. Its predictions come back to be scored on-prem against ground truth labels nobody else has; before this commit the requirement was enforced only by an operator's own key failing to decrypt. The benchmark owner deliberately does not decide any of this. They are not the party at risk, and MedPerf has no way for them to enforce it -- `benchmark_owner` is one of the roles each asset owner may choose to accept, nothing more. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow the key material in the Google Cloud backend: the ciphertext is in GCS, the wrapped key is in GCS, and the wrapping key is in KMS. Google holds everything needed to decrypt any asset, and what stops it is Google enforcing its own IAM against itself. An asset owner who does not want to make that trust has had no alternative. `kbs/` is one: a service the owner runs, holding the key and the encrypted asset, releasing them only to a workload that proves by attestation that it is the script the owner authorized, on the inputs they authorized, in a genuine confidential VM. It enforces the policy MedPerf already writes to Google Cloud. There an asset owner installs an attribute mapping on a workload identity pool and binds IAM principals matching it; here the broker evaluates the same terms against the same attestation claims itself. The identity strings are byte for byte the ones `WorkloadBinding` produces, so an asset moves between backends without its owner restating what they meant. What this does not change, checked against the documentation rather than assumed: a Confidential Space workload cannot obtain raw hardware evidence. The launcher collects the attestation report, has a verifier check it, and hands back a signed token. So a broker still trusts *a* verifier -- it just no longer has to be Google's, since Intel Trust Authority issues tokens for TDX. The trust moves from "Google can read its own KMS" to "Google would have to forge an attestation". Two things the protocol needs and did not have: - `medperf_cc.attestation` verifies a Confidential Space token: the x5c chain against a pinned root, then the claims against what the policy requires. Written on `cryptography` because JWT libraries do not validate x5c. - Challenges. A token is short lived but not single use, so without one an observed token could be replayed until it expired. The broker issues a nonce, requires it inside the token, and burns it. Refusals are always a bare 403 with the same message; the reason goes to the log. A caller that learns *why* it was refused can map the policy by probing. The backend is chosen per asset, so a broker-backed dataset and a KMS-backed model can run together, as long as the benchmark script supports both. A configuration naming no backend is a Google Cloud one, so nothing already configured changes. This is also why the workload is no longer handed the stored configuration: the broker's admin token lives in it, and everything a workload is told travels to the VM as environment the operator can read. Each vault now states what its workload may know, field by field. `medperf-cc` grows a `gcp` extra so a broker deployment carries neither the google-cloud stack nor the MedPerf client. `medperf_cc.testing` mints real tokens against a throwaway authority, because a fake that answered "valid" would test nothing -- these paths exist in order to reject things. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A benchmark result is a number the operator reports, and everybody downstream
has to take their word for it. The one setting where that is avoidable is the
one where a confidential VM already produces a signed statement about itself,
and it was going unused.
Before packing up its results, the workload now writes two files beside them: a
statement naming the hashes of what went in and what came out, and an
attestation token whose nonce is that statement's hash. Together they establish
which script ran, on which inputs, producing exactly these bytes, inside genuine
confidential hardware.
medperf result trust_attestation_root # once, pins the root certificate
medperf result verify -e <execution-id>
No new signing machinery, because the token already carries most of it: the
image digest says which script ran, `env_override` says which inputs it was
pointed at, `swname` and `hwmodel` say it was real hardware. Only the results
hash and the measurements are ours to add.
The script is deliberately not in the statement. It comes from the token's
attested `image_digest`, and a workload self-reporting its own image would be
worth nothing. The data and model hashes are, but as *measurements*: taken
inside the VM on the decrypted inputs, where `env_override` only states what the
operator declared. A verifier that sees them agree does not have to know that
the script checks them.
Nothing else belongs in there. There is no execution id: a proof for one
(script, data, model) applied to another execution of the same three is not a
forgery, and if any of the three differ the hashes already reject it.
Verification is offline and expiry is not checked, in both directions -- the
token's and its certificate chain's. A proof records a run that already
happened; requiring a one-hour token to still be current would make every proof
self-destruct.
The proof lives in its own `integrity_proof` field on the execution rather than
in a corner of `metadata`, so an execution without one is visibly unverified
rather than quietly indistinguishable. A workload that cannot obtain a token
still delivers its results and logs a warning: turning "unverifiable" into
"failed" is the worse trade.
The results hash exists in two places -- here and in the base image, which is
built separately and cannot import this package. `test_producer_contract.py`
loads the producer from source and compares it against the verifier, because a
silent disagreement would mean every proof fails to verify with nothing to say
why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hasan7n
requested a deployment
to
testing-external-code
August 12, 2026 11:12 — with 
GitHub Actions
Waiting
GitHub Actions
Waiting
Contributor
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
hasan7n
requested a deployment
to
testing-external-code
August 14, 2026 10:52 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
hasan7n
requested a deployment
to
testing-external-code
August 14, 2026 18:18 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
hasan7n
requested a deployment
to
testing-external-code
August 15, 2026 07:32 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
hasan7n
requested a deployment
to
testing-external-code
August 16, 2026 12:26 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
A confidential workload writes an integrity proof beside its results, and collecting them stored it locally -- but submitting the results never sent it. The server was left holding a metric with no way to check it, and the proof only existed on the machine that happened to collect it. Read it back at submit time and send it with the results. An execution that produced no proof sends no field at all, so "unverifiable" stays distinguishable from "a proof that failed". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner both started a workload and fetched what it produced, and it chose the destination itself -- `result_config` was the runner's own business because the results were assumed to be the operator's. That assumption is what stops anybody else from ever being the party the results are for. Split the fetching half out into a `result_store` service alongside storage, vault and runner. The runner now launches and watches, and is told where the output goes; the store owns the address, the readiness check and the fetch. On naming: "result collector" already means a *person* everywhere else in MedPerf -- `allowed_result_collectors` names roles, and the policy is about who may receive results. Calling the backend a collector too would make one word mean both the party and the place. The backend is a store. The operator needs the address but never credentials, so `receiver_config` is offered as a plain function of the collector's published settings rather than as a store the operator has to hold. Both sides otherwise build from the identical settings dict -- what separates them is whose machine is running -- so there is no second type here, only a narrower entry point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A user had one CC configuration, and it was the operator's -- which left
no way to say "these are my settings for receiving results" separately
from "this is the machine I run workloads on". They are separate roles,
and the results are written to whoever they are for, who need not be
whoever ran the machine.
`user.cc_operator` and `user.cc_collector` are now the same four questions
asked of whichever role you name, replacing ten near-identical methods and
making a third role an entry rather than a copy-paste.
Settings saved under the old shape are still read as the operator's, so an
existing user is not silently unconfigured; saving rewrites them.
The web UI follows the same shape: one `CC_SECTIONS` table drives both
forms, the template loops over it, one `/settings/edit_cc/{section}` route
saves either, and the JavaScript finds everything by section instead of by
a hardcoded list of element ids -- which was also why it only ever knew
about the operator's five GCP fields. Files renamed from cc_operator.* to
cc_settings.* to match what they now are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…perator Results were encrypted for whoever ran the workload, so a policy naming a "result collector" really named who was allowed to *operate* -- and the two could never be different people. A model owner could not have a benchmark run against somebody else's data and receive the scores. Now both asset owners' policies name who *receives* the results, MedPerf resolves the one party they both named (refusing nobody-in-common and more-than-one, since one key means one choice), and the workload writes to that party's store encrypted for their key. The operator supplies the machine and can neither reach the output nor open it. They record who it was for; the collector picks it up with `download_cc_results` and reports it with the ordinary `result submit`. Server side, `result_collector` is a write-once FK that grants that one party read and PUT on an execution they did not create, and nothing to anybody else. It is validated against the two asset owners rather than taken on the operator's word -- only their keys are published to the other parties, so nobody else could have had results encrypted for them, and without the check an operator could hand a stranger rights here. `/me/results/` lists executions recorded as yours to collect, since only you can open them. Two lookups previously read the certificate listings separately -- one for grants, one for the execution. They are the same question, and a grant pins the hash of the key an execution encrypts for, so they now go through one `certificate_of`. `check_operator_is_allowed` encoded the rule this replaces and is gone. Collection trusts the recorded collector rather than re-deriving it from policies, which can be edited between the run and the collection: the recorded id is what the results were actually sealed for. The chestxray example's four policy files were two identical pairs once policies stopped deciding who operates; they are one pair now. CI runs the whole flow twice, once with each operator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The workload identity was built from the same eight fields in three places: both execution flows and the collection command. That is the one piece of this design that must not drift, because nothing is handed between the operator and the collector -- the collector finds what was written for them by deriving the very same identity from the same recorded facts. Two derivations agree until one changes, and then nothing fails loudly: the rebuilt identity names a storage prefix nothing was written to, so the results just appear not to exist. `ConfidentialRun.resolve` is now the only place it is built, with the store that follows from the same collector. The flows keep what is actually theirs -- the operator has a runner, the collector does not -- and `setup_workload` disappears from both. The test that covered it moves to `tests/cc/test_run.py`, where it can also pin the property that matters: operator and collector derive the same prefix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The endpoint answers with the field wrapped in an object, and returning the wrapper reads as an empty one -- which would surface as a collector who has "not configured anywhere to receive results" rather than as anything pointing at the response shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven things, none of which change what the feature does: `receiver_config` becomes `store_config` -- it was named after the act of receiving rather than after the thing it describes, which is where the results are stored. `download_results` and `fetch_results` sat in one file doing two different jobs under two names that said the same thing. They are now `download_metrics`, which reads what is inside, and `download_result_files`, which leaves it as files. The execution flows call the field `confidential_run` rather than `run`. The user entity no longer reads the pre-split metadata shape. There was no released version writing it, so it was back-compatibility with nothing. `check_operator_setup` and `check_collector_setup` are two plain functions again. Folding them into one that took the thing to build saved four lines and cost more than that to read. `result_collector` is now stated when the execution is created and never afterwards: the client sends it in the POST, and the server holds it read-only on PUT. It no longer matters whether the operator is the collector -- it is recorded either way, so the field means "this is a confidential execution and this is whose results these are" rather than "somebody unusual is collecting". The check restricting it to an asset owner is gone with the PUT that needed policing. Finally, and the reason for the last one: nothing decides anything from the recorded collector any more. The asset owners' policies are what say who results were encrypted for -- they said so before the execution existed -- and the server field exists only to grant that party permission to read and report the execution. `collector_recorded_as` is gone, collecting resolves from policies, and reporting asks `collector_role`, which answers who without paying for their key and their store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The field existed to grant one party read and PUT on an execution they did not create. The dataset owner needs exactly that and already has a permission class, so `IsDatasetOwner` joins the GET and PUT lists and the field goes -- along with the migration that added it, a serializer read-only entry, a client schema field, the POST that set it, and the `/me/results/` filter that surfaced it. Nothing is lost from the split itself: who collects still comes from the asset owners' policies, and the collector still fetches with `download_cc_results`. What changes is that the server no longer holds a per-execution record of who that is, and permits the dataset owner instead. That is coarser -- a model-owner collector on somebody else's dataset would not be covered -- and is enough for the scenarios that exist today. The guard in `medperf result submit` goes too. It refused to report results this user could not read, which mattered only because `medperf run` submits as soon as it finishes; that is the thing to fix, not something to work around here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Running a benchmark validated the requested models against Benchmark.get_models_uids, which the server refuses to a model owner: that listing is the benchmark's whole roster, and reading it would show them their competitors. So a model owner could not run their own model through `medperf run` at all. Ask the caller's own approved associations first. For the models a model owner owns that answers the same question, and those are the only ones they could run anyway. The benchmark's roster is read only when the request reaches beyond them, which is the data-owner case and where the listing is readable. Also fixes the singular/plural branches of the error message, which were the wrong way round, and drops a per-model Model.get() round trip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hasan7n
requested a deployment
to
testing-external-code
August 25, 2026 19:58 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
hasan7n
requested a deployment
to
testing-external-code
August 26, 2026 07:58 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
hasan7n
requested a deployment
to
testing-external-code
August 26, 2026 17:55 — with
GitHub Actions
Waiting
GitHub Actions
Waiting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
refactor CC into a separate package and: