Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/spec/analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -1323,8 +1323,10 @@ widening; the two modules do not define a second affine graph representation.
parameter is already fixed to one integer point. An empty parameter context
MUST count as zero; a non-empty context with an unbounded parameter MUST
report unknown.
- `IterationScope.trips()` MUST fix child and parent domains to the same parameter
point before dividing, and take the maximum of those ratios.
- `IterationScope.trips()` MUST count a loop off its bounds, not its domain:
`max(1, ceil(max(stop - start) / step))`, refusing an unbounded span.
`trips_precision` is `EXACT` where that span reaches one value and
`WIDENED` where it reaches several, the count then being an upper bound.

### 2.2 Target-selected Analyzers

Expand Down
16 changes: 7 additions & 9 deletions docs/spec/code-organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ truth for the directory's structure and invariants.
|---|---|---|
| `ir/core/` | [core-ir](./core-ir.md) | Shared node algebra: `Module` / `Expr` / `Var` / `Constant` / `Tuple` / `Op` / `Call` / `Stmt` (base class) / `OpSchema` / `ParamDef` / call-graph and ownership queries / typed metadata attach-detach and diagnostics / `@register_op` / `@register_alias` / `op_registry` / `errors`. |
| `ir/types/` | [types](./types.md) | Type-system root: `Type` / `TensorType` / `TupleType` / `UnitType` / `CallableType` / `DType` / `StorageKind` / `resolve_storage` / local projections (`local_type_of`) / tensor-leaf, byte-by-storage, and topology-extent queries / `dim.*` (with their typeinfer). |
| `ir/types/shard/` | [shard](./shard.md) | Shard / layout sublayer: `Topology` / `Mesh` / `Layout` / `ComposedLayout` / `ShardLayout` / `ShardAttr` (`Split` / `Broadcast` / `Dynamic` / `Partial`). The physical nesting reflects the spec's "sublayer" relationship. |
| `ir/types/{int_tuple,stride,layout,layout_algebra,shard_layout,mesh}.py` | [shard](./shard.md) | `Topology` / `Mesh` / `Layout` / `ComposedLayout` / `ShardLayout` / `ShardAttr` (`Split` / `Broadcast` / `Dynamic` / `Partial`), filed as CuTe files them: int tuples (`flatten` / `unflatten` / `repeat_like` / `product`), strides (`compact_major` / `idx2crd` / `crd2idx`), layouts and the algebra over them each in their own module. |
| `ir/mesh_scope.py` | [shard](./shard.md) | Which scope a statement stands inside and what it admits: `merge_mesh`, `device_layout`, `covered_by_scope`, `check_topology`. Neither a type nor a visitor, so it sits beside `ir/isl_interop.py` rather than in either. |
| `ir/constraints/` | [parser](./parser.md) | Authored `where(layout=..., mesh=..., storage=...)` constraint records: the shared base plus layout, mesh, and storage constraints, attached by the parser and read back by the Python printer. |
| `ir/visitor.py` | [visitor-mutator](./visitor-mutator.md) | `ExprFunctor` / `ExprVisitor` / `ExprWalker` / `ExprCollector` / `ExprCloner` / `BindingSubstitutionCloner` / `StmtVisitor` / `StmtMutator` / `StmtExprMutator`, plus `collect_exprs`, value-operand/function-value queries, and the canonical `PrimFunction` walk and rewrite entries. |
| `ir/isl_interop.py` | [types](./types.md) | Interoperation between dimension and shape IR values and isl: expression rendering and decoding, normalization, value ranges, and shape-domain construction. Pure isl operations remain in `utils/isl_utils.py`. |
Expand Down Expand Up @@ -74,14 +75,11 @@ physical directory layout reflects that boundary directly.
**Reading notes:**

- `ir/` holds the IR proper and its sublayers only. `ir/types/` is the
root of the type system; `ir/types/shard/` is its shard / layout
sublayer ([architecture §3](./architecture.md#3-type-system)). The
physical nesting reflects the spec's conceptual "sublayer".
- The placement of `shard/` under `types/` is a filing decision, not a
consumer restriction: `Topology` / `Mesh` / `Layout` / `ShardLayout`
are consumed directly by `parser`, `tir`, and `codegen`. The
hierarchy expresses "role in the type system", not "who may import
it".
root of the type system, and the shard / layout types sit in it
directly ([architecture §3](./architecture.md#3-type-system)): they
are types, and a folder naming them a sublayer said nothing a reader
could use. Their filing follows CuTe's, one module per thing the
algebra is written against.
- `codegen/` and `parser/` sit outside `ir/`. By the
[architecture §1](./architecture.md#1-spec-relationship-map)
pipeline they are the front-end producer and back-end consumer of
Expand Down
13 changes: 10 additions & 3 deletions docs/spec/hir.md
Original file line number Diff line number Diff line change
Expand Up @@ -663,12 +663,14 @@ Tensor structural operations; consensus ops (`Transpose` / `Slice` / `Concat`
([torch tensor manipulation ops](https://pytorch.org/docs/stable/torch.html#indexing-slicing-joining-mutating-ops)).

`Transpose`, statically positioned `Slice`, and `Reshape` derive a view layout from
their input when it states one. An input with `layout=None` produces a view with
`layout=None`. Neither case says that the view materialized.
their input when it states one. For `Slice` and `Reshape`, an input with
`layout=None` produces a view with `layout=None`. Neither case says that the
view materialized.

- `Transpose` MUST permute the layout shape and strides by the same permutation
as the tensor shape. A `ShardLayout` MUST remap its split positions through
the registered relation.
the registered relation. An input with `layout=None` MUST permute the C order
it stands for and state the result as a `Layout`.
- `Slice` is normalized as `Slice(x, starts, sizes=..., strides=...)`.
`starts` is a tuple of rank-0 integer operands; `sizes` and `strides` are
`ShapeDim` attributes stored in the same IR normal form as every other dim.
Expand All @@ -681,6 +683,11 @@ their input when it states one. An input with `layout=None` produces a view with
offset is the source offset plus the starts multiplied by the source strides,
and its outer layout carries the sliced shape and retained strides (multiplied
by any slice step).
- A source axis written as a group of modes has no single stride: its start
MUST be read through the group as a mixed-radix coordinate, and the window
MUST be the group's least-stepping modes, as many as its size takes, in the
order the group wrote them. A start that is not a multiple of the size, a
size landing on no mode boundary, and a step MUST each be refused.
- A `ShardLayout` slice MUST preserve its mesh attributes when every narrowed
logical axis is unsplit. The corresponding primitive layout position takes
the window size and stepped stride. Static starts wrap that shard layout in a
Expand Down
11 changes: 6 additions & 5 deletions docs/spec/inspection.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,12 +185,13 @@ placement sugar of [parser §2.1](./parser.md#21-syntax), in both type slots and
op-attribute slots, when every mesh identifier in that sugar has an explicit
scope binding. A binding is either a `with <mesh> as <name>` region or the
function's own execution domain. A mesh merely restated in another expression
is not a binding. Without such a binding, the printer MUST use the verbose
`ShardLayout(...)` form rather than inventing a name.
is not a binding, and the printer never invents a name for one.

Placement sugar states the layout's own dimensions with each `Split` written on
the dimension it divides, adds the stride tuple whenever the layout has one,
and states the remaining mesh axes in a `{axis @ ...}` set. Because the parser
Placement sugar states one dimension per tensor axis, with each `Split` written
on the dimension it divides, adds the stride tuple whenever the layout has one,
and states the remaining mesh axes in a `{axis @ ...}` set. A layout that groups
an axis's modes has no sugar; elsewhere a group is written as the shape tuple
it is. Because the parser
reads an unstated mesh axis as `Broadcast`, the set carries every `Partial` and
carries `Broadcast` only when no `Split` or `Partial` would otherwise name the
mesh.
Expand Down
6 changes: 6 additions & 0 deletions docs/spec/parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ inside a region but bound outside it are captured as `MeshRegion.args`, with a
fresh `MeshRegion.params` binding used by the body. Capture is performed one
region boundary at a time, so nested regions pass a value through each door.

A loop body holds `with Mesh(...)` statements, and the loop carries what one
binds. Because the body repeats, a name the `with` reads on its way to binding
it escapes as a name read after it does, and the loop carries both. A TIR loop
bound that is not a literal is read as the dimension arithmetic the loop was
lowered from, so a bound naming a mesh coordinate reads back as it was printed.

## 2. Syntax and Rules

### 2.1 Syntax
Expand Down
70 changes: 42 additions & 28 deletions docs/spec/shard.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ class Mesh:

Attributes:
topologies: attribute; Ordered topology sequence.
layout: attribute; Mesh layout or constant sliced layout.
layout: attribute; One nested arrangement, mode i being level i.
names: attribute; Human-readable layout-axis names.
"""

Expand All @@ -303,42 +303,51 @@ class Mesh:
Field meanings:

- `topologies` — the ordered device-domain descriptions (`name` + `size`)
- `layout` — the mesh's own shape / strides (a `Layout`); a constant slice
(`m[...]`) replaces it with a `ComposedLayout` recording the sub-box
([tir §1.5](./tir.md#15-sync))
- `layout` — a nested `Layout` whose mode `i` is level `i`'s own arrangement; a
constant slice (`m[...]`) replaces it with a `ComposedLayout` recording the
sub-box ([tir §1.5](./tir.md#15-sync))
- `names` — optional human-readable names (`cta.x`, `cta.y`, …)

`Mesh` describes the parallel device domain; it is not a tensor layout
object.
Every mesh MUST state one arrangement per level it names, each in that level's
own numbering, as one nested `Layout` whose mode `i` is level `i` -- a mesh
naming one level included. `Mesh` normalizes what was written into that form at
construction: one arrangement over all the levels' axes is cut at the level
boundaries, each level taking axes left to right until their extents multiply to
its own size and its steps divided by what the levels under it hold. An axis of
extent one adds no positions and joins the level being filled. A boundary no
prefix of axes lands on MUST be refused.

Because the layout is a nested `Layout`, the layout operations read it: `get(mesh.layout, i)`
is level `i`'s own arrangement, `flatten(mesh.layout)` is every level's axes end
to end, and `repeat_like` against `mesh.layout.shape` states per axis whatever is
stated per level. `device_layout(mesh)` is those axes as the device numbers its
positions, each stepping by what its level states times what the levels under it
hold, and a slice's `ComposedLayout.offset` is where the whole mesh starts in
that numbering.

Mesh composition uses the following rules:

- `composed(meshes)` MUST replace all levels in force when the inner mesh names
every current level, and MUST concatenate meshes whose level-name sets are
disjoint. For concatenation, each outer stride and offset is scaled by the
product of positions in the levels below it:
`offset = outer_offset * below + inner_offset`.
- A sliced mesh is composable; its slice offset remains a
`ComposedLayout` offset after composition. A mesh whose level names partially
overlap the levels in force MUST be rejected rather than decomposed.
- `composed(meshes)` invokes `check_topology` on its result. For each named
level with a concrete declared extent, its projected position count MUST NOT
exceed that extent; symbolic extents are deferred until dimensions are bound.
This check does not reject same-level nesting, because replacement determines
the final position count.
- `merge_mesh(meshes)` MUST `append` a mesh whose level names are disjoint from
those in force, MUST replace them entirely when the inner mesh names every
one, and MUST `replace` the trailing levels when the inner mesh names a
suffix of them, keeping the levels above and their names unchanged. Level
names overlapping in any other way MUST be rejected rather than decomposed.
- `append` and `replace` concatenate the per-level arrangements; no stride or
offset is rescaled, because each level already states its own numbering.
- `merge_mesh(meshes)` invokes `check_topology` on its result. For each named
level with a concrete declared extent, its position count MUST NOT exceed
that extent; symbolic extents are deferred until dimensions are bound. A
level stating a run is already bounded by `Mesh.__getitem__` and is not
checked again.

HIR `MeshRegion` applies this composition only at its body boundary. Its `args`
are evaluated in the enclosing scope and are not recomposed merely because the
value is consumed by a region.

`Mesh` MAY carry more than one `Topology` (e.g. `warp(4) x thread(32)`); the
full sequence is always `topologies`.

- constraints:
- `Topology` construction rejects a `None` size. `Mesh` construction rejects
a `None` entry in its layout shape. Beyond that explicit-extent check,
`Mesh` is a frozen record: it performs no construction-time normalization
or position-consistency check. Its `topologies` field is a
a `None` entry in its layout shape and normalizes the layout into one mode
per level; beyond that it performs no position-consistency check. Its `topologies` field is a
`tuple[Topology, ...]`; helpers such as `make_mesh` construct that tuple for
handwritten Python.
- The author surface is `with Mesh(("cta",), layout=(128,)) as cta:`. The
Expand All @@ -358,8 +367,9 @@ full sequence is always `topologies`.
- The layout of the positions one level has is the axes up to and including
that level's segment, with their strides divided by the product of the sizes
of the levels below it; a stride that division does not divide exactly MUST
be refused. A Mesh naming one level states its own layout and is not
projected. A Mesh naming several MUST NOT also be sliced.
be refused. A Mesh naming one level takes every axis into its single mode,
which needs no division and therefore no declared extent. A Mesh naming
several MUST NOT also be sliced.
- Nested single-level Mesh scopes compose to exactly that shape: the axes join
outermost first and each outer stride is scaled by the positions below it.
A value distributed at two levels at once may therefore be written either
Expand All @@ -385,7 +395,7 @@ full sequence is always `topologies`.
The placed-layout constructor has one additional guard: a single layout may
split a named level only once. If two distinct meshes used by one placed layout
name the same topology level, parsing MUST reject that layout at its source
node. This is a layout-construction rule, independent of `composed()`'s
node. This is a layout-construction rule, independent of `merge_mesh()`'s
scope-composition rules.

### 5.1 `Placement`
Expand Down Expand Up @@ -762,6 +772,10 @@ copy that displacement to a materialized consumer.

## 9. Layout construction and mesh-scope projection

Making the steps of a compact arrangement and reading an index back into the
coordinate that reaches it are not operations on layouts, so they are filed
apart from the algebra, as CuTe files `stride.hpp` apart from `layout.hpp`.

```python
class NotProjectable(ValueError):
"""Report that a layout cannot serve as a mesh execution scope."""
Expand Down
3 changes: 3 additions & 0 deletions docs/spec/tir.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,9 @@ class PrimFunction(Stmt):
- **`MeshScope` mesh in scope**. Any embedded `ShardLayout` MUST
reference a mesh on the active TIR `MeshScope` traversal cache or a parameter's
`ShardLayout.mesh`.
- **`For` bound coordinates**. A `MeshCoord` read by `For.start` / `.stop` /
`.step` MUST name a literal in-range axis of a mesh bound by an enclosing
`MeshScope`.
- **`Evaluate.callable`**. When `callable` is a `SymbolRef`
([§2.1](#21-symbolref)), module-level resolution MUST find exactly one
`PrimFunction` of that name in the enclosing `Module`, `args` length
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial/showcase.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
}
},
"outputs": [],
"source": "from __future__ import annotations\n\nimport ast\nimport math\nimport re\nimport sys\nfrom pathlib import Path\n\nfrom tilefoundry import func, module\nfrom tilefoundry.analysis import analyze as run_analysis\nfrom tilefoundry.dsl import ConstTensor, DimVar, DimVarRangePat, Mesh, Tensor, tf\nfrom tilefoundry.dsl.tf import * # noqa: F401, F403 - bare tile() in the fused body\nfrom tilefoundry.inspection.analysis_report import render_analysis, render_text\nfrom tilefoundry.ir.types.shard import Topology\nfrom tilefoundry.target import CudaTarget\n\nHIDDEN = 256\nQUERY_HEADS = 8\nKV_HEADS = 2\nHEAD_DIM = 32\nKV_DIM = KV_HEADS * HEAD_DIM\nGQA_GROUP = QUERY_HEADS // KV_HEADS\nROPE_CONTEXT = 8192\nCTX = DimVar(\"ctx_len\", 1, ROPE_CONTEXT + 1)\nSCALE = 1.0 / math.sqrt(HEAD_DIM)\nWORKERS = 4\nBLOCK = 128\n\n_H200 = CudaTarget(\"nvidia.h200_sxm\")\n_CTA = Topology(\"cta\", 132)\n"
"source": "from __future__ import annotations\n\nimport ast\nimport math\nimport re\nimport sys\nfrom pathlib import Path\n\nfrom tilefoundry import func, module\nfrom tilefoundry.analysis import analyze as run_analysis\nfrom tilefoundry.dsl import ConstTensor, DimVar, DimVarRangePat, Mesh, Tensor, tf\nfrom tilefoundry.dsl.tf import * # noqa: F401, F403 - bare tile() in the fused body\nfrom tilefoundry.inspection.analysis_report import render_analysis, render_text\nfrom tilefoundry.ir.types import Topology\nfrom tilefoundry.target import CudaTarget\n\nHIDDEN = 256\nQUERY_HEADS = 8\nKV_HEADS = 2\nHEAD_DIM = 32\nKV_DIM = KV_HEADS * HEAD_DIM\nGQA_GROUP = QUERY_HEADS // KV_HEADS\nROPE_CONTEXT = 8192\nCTX = DimVar(\"ctx_len\", 1, ROPE_CONTEXT + 1)\nSCALE = 1.0 / math.sqrt(HEAD_DIM)\nWORKERS = 4\nBLOCK = 128\n\n_H200 = CudaTarget(\"nvidia.h200_sxm\")\n_CTA = Topology(\"cta\", 132)\n"
},
{
"cell_type": "markdown",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial/showcase.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ from tilefoundry.analysis import analyze as run_analysis
from tilefoundry.dsl import ConstTensor, DimVar, DimVarRangePat, Mesh, Tensor, tf
from tilefoundry.dsl.tf import * # noqa: F401, F403 - bare tile() in the fused body
from tilefoundry.inspection.analysis_report import render_analysis, render_text
from tilefoundry.ir.types.shard import Topology
from tilefoundry.ir.types import Topology
from tilefoundry.target import CudaTarget

HIDDEN = 256
Expand Down
19 changes: 1 addition & 18 deletions src/tilefoundry/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,24 +37,7 @@
from tilefoundry.ir.core.pattern import DimVarRangePat, Pattern
from tilefoundry.ir.types import DType, TensorType, TupleType, Type
from tilefoundry.ir.types.dim import DimVar
from tilefoundry.ir.types.shard import (
B,
Broadcast,
ComposedLayout,
Swizzle,
Dynamic,
IntTuple,
Layout,
LayoutBase,
Mesh,
P,
Partial,
S,
ShardAttr,
ShardLayout,
Split,
Topology,
)
from tilefoundry.ir.types import B, Broadcast, ComposedLayout, Dynamic, IntTuple, Layout, LayoutBase, Mesh, P, Partial, S, ShardAttr, ShardLayout, Split, Swizzle, Topology


from tilefoundry.ir.tir.stmt import Stmt
Expand Down
3 changes: 1 addition & 2 deletions src/tilefoundry/analysis/access.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@
from tilefoundry.ir.hir.tensor.slice import Slice
from tilefoundry.ir.isl_interop import index_set
from tilefoundry.ir.types import TensorType
from tilefoundry.ir.types.shape_helpers import static_dim_value
from tilefoundry.ir.types.utils import local_type_of
from tilefoundry.ir.types.utils import local_type_of, static_dim_value
from tilefoundry.utils.isl_utils import cardinality, has_unbounded_param
from tilefoundry.visitor_registry.access_relation import (
BoundaryRelation,
Expand Down
7 changes: 2 additions & 5 deletions src/tilefoundry/analysis/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,14 @@
is_concrete,
specialize_concretely,
)
from tilefoundry.ir.types import callable_type_for
from tilefoundry.ir.types.shape_helpers import static_dim_value
from tilefoundry.ir.types.shard import (
Topology,
)
from tilefoundry.ir.types import Topology, callable_type_for
from tilefoundry.ir.types.substitute import (
DimSubstitutionError,
dim_vars_by_name,
substitute_shape_dim,
substitute_topology_dims,
)
from tilefoundry.ir.types.utils import static_dim_value
from tilefoundry.ir.visitor import BindingSubstitutionCloner, collect_exprs
from tilefoundry.target import UnsupportedCapabilityError
from tilefoundry.target.facts import TopologyFacts
Expand Down
Loading
Loading