feat: check distributed HIR against explicit references

[UranusSeven]

TileFoundry's logical evaluator runs sharded contractions on full tensors, which cannot expose a missing cross-device reduction. This change adds check --reference SOURCE --distributed: the candidate executes on simulated rank-local tensors with explicit collectives, then its reconstructed outputs are compared with the selected reference HIR using the existing numerical predicates.

• Add AllReduce, AllGather, and ReduceScatter over existing device-mesh axes, with ownership and participation contracts.
• Share logical activations and weights between reference and candidate; validate state outputs, repeated state updates, subgroup isolation, dynamic dimensions, and empty partitions.
• Reject incomplete partial outputs, implicit communication through Reshard, and unsupported distributed operations.
• Include the engine-scope RFC, finalized implementation plan, owner specifications, and an executed tutorial available through tilefoundry tutorial distributed-check.

This is the correctness foundation for engine-scope optimization. Supported layouts use equal contiguous partitions on a single device topology level. Distributed memory/communication costing, ragged expert exchange, physical collective lowering, and strategy search remain follow-up work. All existing analyze selectors explicitly reject uncosted collectives.

Validation

CPU suite: 475 passed, 1 skipped, 1 deselected.

python -m pytest tests/evaluator tests/ops/ir \
  tests/parser/test_mesh_visibility.py tests/ir/types/test_mesh.py \
  tests/ir/types/test_shard_layout.py tests/ir/test_shard_layout_local_shape.py \
  tests/ir/test_function_call_typeinfer.py tests/cli tests/analysis \
  -q -k 'not test_a_pinned_extent_on_a_root_that_reaches_a_child'

One existing cross-device test is skipped on CPU, and the existing CLI test named above is excluded because it selects CUDA by default. Physical GPU execution was not tested.

Ruff, specification/reference checks, comment/annotation checks, language/path checks, finalized-plan checks, and git diff --check pass. The tutorial notebook was executed and rendered with passing comparison results; all four analysis selectors were checked for explicit rejection of missing collective costs.

[zhen8838]

@UranusSeven Thanks for this — the RFC and the rank-local evaluator are good work, and the evaluator is close to what issue #138 has been asking for.

I noticed the PR was closed shortly after opening, so I don't know whether you're reworking it. Either way I'd like to discuss a few points before either of us builds further on top: unifying the distributed value types with ShardTensor (#162, #165, #174 landed some of this groundwork), dropping --distributed in favour of the evaluator deciding for itself, and whether search and evaluation can face only reshard with collectives chosen at materialization.

Opened a discussion for it: #181

[UranusSeven]

@UranusSeven Thanks for this — the RFC and the rank-local evaluator are good work, and the evaluator is close to what issue #138 has been asking for.

I noticed the PR was closed shortly after opening, so I don't know whether you're reworking it. Either way I'd like to discuss a few points before either of us builds further on top: unifying the distributed value types with ShardTensor (#162, #165, #174 landed some of this groundwork), dropping --distributed in favour of the evaluator deciding for itself, and whether search and evaluation can face only reshard with collectives chosen at materialization.

Opened a discussion for it: #181

Thanks for your reply! This PR was a demo for a quick en2end experiment, not ready for reviewing & merge yet. That's why I closed it.

Totally agree with dropping --distributed and make the evaluator a unified entity.

Let's move on with #181