TorchConverter: states=[] cannot express "this model has no states"

[john-rocky]

Summary

ct.convert(..., states=[]) and ct.convert(...) are indistinguishable inside
TorchConverter, so a caller converting an ExportedProgram has no way to say "do not
turn my mutable buffers into states". The empty list is normalised into the same value as
the default before the decision is made.

coremltools/converters/mil/frontend/torch/converter.py, in TorchConverter.__init__
(9.0, lines 579-582):

        # process states
        if states is None:
            states = []
        self.states = states

and then, on the ExportedProgram path (line 640):

            if states is None or len(states) == 0:
                # For torch.export, we default to create states from torch mutable buffers
                self.states = []
                for name, tensor in self.graph.buffers.items():
                    ...
                    self.states.append(state)

By the time that runs, states is None can never be true — it was rebound to [] above —
so the condition reduces to len(states) == 0, and an explicit empty list takes the
inference branch. Every mutable buffer becomes a state whatever the caller asked for.

Why it matters

ExecuTorch's Core ML backend has a take_over_mutable_buffer=False option, and cannot
honour it. Its converter call has no way to express the intent, and the models it produces
then require an MLState the runtime was not built to supply, failing at execute rather
than at conversion (pytorch/executorch#21855). The comment in the ExecuTorch source says
the option exists precisely because the OSS runtime does not handle state.

Suggested change

Keep the caller's intent before normalising:

        # process states
        # An empty list is the caller saying the model has no states, which is not the
        # same as saying nothing; only the latter asks for them to be inferred.
        infer_states_from_buffers = states is None
        if states is None:
            states = []
        self.states = states

and use infer_states_from_buffers in place of states is None or len(states) == 0.

Callers passing None or a non-empty list are unaffected. The one behaviour that changes
is states=[], which today means "infer them" and would come to mean what it reads as.

What I did not do

I could not reduce the ExecuTorch symptom below a full LLM export, so this report is from
reading the conversion path rather than from a minimal end-to-end repro. The two code
paths above are quoted verbatim from the installed 9.0 source, and the ExecuTorch side is
in pytorch/executorch#21855 with its own detail.

Environment: coremltools 9.0, torch 2.13.0, macOS arm64.

[john-rocky]

I have since been able to build the end-to-end case, and it turns out the change I
suggested above is not sufficient. Correcting that here rather than leaving a
suggestion that does not work.

The repro I could not produce before

executorch.extension.llm.export.export_llm with a Core ML backend and
enable_state: True now builds and loads. It fails at execute:

[backend_delegate.mm:435] [Core ML] Model execution failed
  The input feature for layers_7_conv_conv_state must be an MLState, but it was not.
[method.cpp:1530] CALL_DELEGATE execute failed at instruction 0: 0x32

That is the ExecuTorch symptom (pytorch/executorch#21855), reproduced on the host with
LFM2.5-350M, so it no longer needs a device.

What happens when states=[] is honoured

I applied the change above locally — keeping the caller's intent before the None is
normalised, and using it in place of states is None or len(states) == 0 — and had
ExecuTorch pass states=[] alongside its existing take_over_mutable_buffer=False.
Conversion then fails earlier instead:

ValueError: Torch var layers.0.conv.conv_state not found in context

Which makes sense in hindsight. Declining to build states does not tell the converter what
those mutable buffers should be instead; the graph still refers to them, and nothing
binds them. So honouring the empty list needs a second half — mutable buffers handled as
ordinary tensors, presumably surfaced as an input and an output — and that is a larger
change than the one I proposed.

I am leaving the report open because the asymmetry is still real: states=[] reads as
"no states" and is silently treated as "work them out for me", and a caller that has a
runtime without state support has no way to say so. But the fix is not two lines, and I
did not want that sitting here unchallenged.

Environment unchanged: coremltools 9.0, torch 2.13.0, macOS arm64.