Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions THIRD-PARTY-LICENSES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2741,9 +2741,9 @@ limitations under the License.
** itoa; version 1.0.18 -- https://crates.io/crates/itoa
** libc; version 0.2.189 -- https://crates.io/crates/libc
** manyhow-macros; version 0.11.4 -- https://crates.io/crates/manyhow-macros
** openjd-expr; version 0.9.0 -- https://crates.io/crates/openjd-expr
** openjd-model; version 0.9.0 -- https://crates.io/crates/openjd-model
** openjd-sessions; version 0.7.0 -- https://crates.io/crates/openjd-sessions
** openjd-expr; version 0.10.0 -- https://crates.io/crates/openjd-expr
** openjd-model; version 0.10.0 -- https://crates.io/crates/openjd-model
** openjd-sessions; version 0.7.1 -- https://crates.io/crates/openjd-sessions
** pin-project-lite; version 0.2.17 -- https://crates.io/crates/pin-project-lite
** portable-atomic; version 1.15.0 -- https://crates.io/crates/portable-atomic
** proc-macro2; version 1.0.107 -- https://crates.io/crates/proc-macro2
Expand Down
6 changes: 3 additions & 3 deletions rust-bindings/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ name = "_openjd_rs"
crate-type = ["cdylib", "rlib"]

[dependencies]
openjd-expr = "0.9.0"
openjd-model = "0.9.0"
openjd-sessions = "0.7.0"
openjd-expr = "0.10.0"
openjd-model = "0.10.0"
openjd-sessions = "0.7.1"
tokio = { version = "1", features = ["rt-multi-thread"] }
uuid = { version = "1", features = ["v4"] }
serde_json = "1"
Expand Down
32 changes: 22 additions & 10 deletions rust-bindings/src/model/template_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -979,19 +979,25 @@ pub(crate) struct PyAmountRequirement {
impl PyAmountRequirement {
#[new]
#[pyo3(signature = (*, name, min=None, max=None))]
fn new(name: String, min: Option<PyFormatString>, max: Option<PyFormatString>) -> Self {
fn new(name: PyFormatString, min: Option<PyFormatString>, max: Option<PyFormatString>) -> Self {
Comment thread
leongdl marked this conversation as resolved.
PyAmountRequirement {
inner: AmountRequirement {
name,
name: name.inner,
min: min.map(|fs| fs.inner),
max: max.map(|fs| fs.inner),
},
}
}

/// `@fmtstring` as of openjd-rs#409: a name may carry expressions, and its
/// §3.3.1.1 constraints are checked on the resolved value. Use `.raw()` for
/// the template text. The job-side `AmountRequirement.name` is the resolved
/// `str`.
#[getter]
fn name(&self) -> &str {
&self.inner.name
fn name(&self) -> PyFormatString {
PyFormatString {
inner: self.inner.name.clone(),
}
}

#[getter]
Expand All @@ -1011,7 +1017,7 @@ impl PyAmountRequirement {
}

fn __repr__(&self) -> String {
format!("AmountRequirement(name={:?})", self.inner.name)
format!("AmountRequirement(name={:?})", self.inner.name.raw())
}

#[allow(clippy::type_complexity)]
Expand Down Expand Up @@ -1054,22 +1060,28 @@ impl PyAttributeRequirement {
#[new]
#[pyo3(signature = (*, name, any_of=None, all_of=None))]
fn new(
name: String,
name: PyFormatString,
any_of: Option<Vec<PyFormatString>>,
all_of: Option<Vec<PyFormatString>>,
) -> Self {
PyAttributeRequirement {
inner: AttributeRequirement {
name,
name: name.inner,
any_of: any_of.map(|v| v.into_iter().map(|fs| fs.inner).collect()),
all_of: all_of.map(|v| v.into_iter().map(|fs| fs.inner).collect()),
},
}
}

/// `@fmtstring` as of openjd-rs#409: a name may carry expressions, and its
/// §3.3.2.1 constraints are checked on the resolved value. Use `.raw()` for
/// the template text. The job-side `AttributeRequirement.name` is the
/// resolved `str`.
#[getter]
fn name(&self) -> &str {
&self.inner.name
fn name(&self) -> PyFormatString {
PyFormatString {
inner: self.inner.name.clone(),
}
}

#[getter]
Expand Down Expand Up @@ -1103,7 +1115,7 @@ impl PyAttributeRequirement {
}

fn __repr__(&self) -> String {
format!("AttributeRequirement(name={:?})", self.inner.name)
format!("AttributeRequirement(name={:?})", self.inner.name.raw())
}

#[allow(clippy::type_complexity)]
Expand Down
21 changes: 21 additions & 0 deletions specs/python-expr-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,27 @@ never raises (Python convention). This makes unresolved values
safe to inspect in debuggers and tracebacks while still failing
loudly anywhere a real value is expected.

**How an unresolved operand propagates.** Three rules are worth stating,
because each decides whether an expression errors or concludes unresolved,
and all three changed in openjd-expr 0.10.0 (openjd-rs#407):

* A conditional whose test is unresolved evaluates both branches. A *value*
error in one branch is absorbed — run time may select the healthy one — but
a memory or operation *budget* exceedance propagates, because the budget was
spent in this evaluation whichever branch run time takes.
* `and` / `or` behave the same way for operands after the first unresolved
one: value errors are suppressed for the same short-circuit reason, budget
exceedances are not.
* A list comprehension whose filter evaluates unresolved concludes
`unresolved[list[T]]` rather than erroring, whether the iterable is
unresolved or concrete. Per-element inclusion is undecidable, so the
elements accumulated so far are abandoned. `T` is the body's type derived
under an *unresolved* loop variable, so evaluating the body on an element the
run-time filter may exclude cannot raise a spurious value error:
`[10 // x for x in [0, 2] if x > N]` with `N` unresolved is
`unresolved[list[int]]`, not a division-by-zero. A filter whose type can
never be a boolean is still an error.

### `SymbolTable`

Hierarchical key-value store providing variable bindings for expression
Expand Down
71 changes: 59 additions & 12 deletions specs/python-model-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -795,9 +795,10 @@ cancel.notify_period_in_seconds # Optional[int]
Job-time host requirements. Distinct from the template-time
``HostRequirements`` / ``AmountRequirement`` / ``AttributeRequirement``
(see `openjd.model._v1.template`): the template-time variants carry
unresolved ``FormatString`` values for ``min`` / ``max`` / ``anyOf`` /
``allOf``, while these job-time variants carry the post-``create_job``
resolved ``f64`` (amounts) and ``str`` (attributes) values.
unresolved ``FormatString`` values for ``name``, ``min`` / ``max`` / ``anyOf``
/ ``allOf``, while these job-time variants carry the post-``create_job``
resolved ``str`` name and the resolved ``f64`` (amounts) and ``str``
(attributes) values.

```python
hr = step.host_requirements # alias: step.hostRequirements
Expand Down Expand Up @@ -955,16 +956,38 @@ hr.amounts # Optional[list[AmountRequirement]]
hr.attributes # Optional[list[AttributeRequirement]]

amt = hr.amounts[0]
amt.name # str
amt.name # FormatString — raw, may hold expressions
amt.min # Optional[FormatString]
amt.max # Optional[FormatString]

attr = hr.attributes[0]
attr.name # str
attr.name # FormatString — raw, may hold expressions
attr.any_of # Optional[list[FormatString]] (alias: anyOf)
attr.all_of # Optional[list[FormatString]] (alias: allOf)
```

`name` is `@fmtstring` (§3.3.1, §3.3.2): `"{{Param.FleetAttribute}}"` is a
valid template-time name, and the §3.3.1.1 / §3.3.2.1 capability-name
constraints apply to the *resolved* name. Which stage applies them depends on
when the name's value becomes known:

| Name | Checked at | Applied |
|---|---|---|
| literal | decode | pattern, length, reserved scopes, uniqueness, standard-capability values |
| fully static — `{{ 'attr.custom.x' }}`, or built only from `let` bindings with literal values | decode | the same set, against the resolved text |
| partly static — `"amount.custom.<95 chars>{{Param.X}}"` | decode | the length *lower bound*: `resolves to at least 109 characters, exceeding the maximum of 100.` |
| parameter-dependent | `create_job` | pattern, length, reserved scopes, case-insensitive uniqueness within `amounts` and within `attributes`, and the standard-capability value rules the resolved name selects |

A name is resolved at job creation, so only symbols available there are in
scope for it: `Task.Param.*` in a name is rejected at decode as an undefined
variable even where the same symbol is valid elsewhere in the step.

`name` is an `openjd.expr.FormatString`, like every other FormatString-typed
field on the template types, so reading the template text needs `.raw()` and
constructing one needs `FormatString(...)` rather than a bare `str`. On 0.9.0
it was a `str` in both directions. The job-side `AmountRequirement.name` /
`AttributeRequirement.name` are still `str`: they hold the resolved name.

### `StepDependency`

```python
Expand Down Expand Up @@ -1372,25 +1395,49 @@ template = decode_job_template(template={...}, supported_extensions=["EXPR"])

# 2. Read the template's declared profile back out.
profile = template.profile # ModelProfile(revision=V2023_09, extensions=[EXPR])
profile.revision # SpecificationRevision.V2023_09
profile.revision # SpecificationRevision.v2023_09
profile.extensions # [ModelExtension.EXPR]
profile.has_extension(ModelExtension.EXPR) # True

# 3. Build it manually if needed (e.g. when validating against a different
# policy than the template declared).
# 3. Build it manually if needed (e.g. to enable an extension the template
# does not declare, or to carry caller limits).
manual = ModelProfile(extensions=[ModelExtension.EXPR, ModelExtension.TASK_CHUNKING])
ModelProfile.from_strings(SpecificationRevision.V2023_09, ["EXPR"])
ModelProfile.from_strings(SpecificationRevision.v2023_09, ["EXPR"])

# 4. Pass to create_job through a ValidationContext if you want to
# override the template's default validation context.
# 4. Pass to create_job through a ValidationContext, usually to attach
# caller limits. The context must COVER the template: same revision, and
# every extension the template declares (enabling more is fine).
limits = CallerLimits(max_step_count=100, max_task_count=10_000)
ctx = ValidationContext(profile, caller_limits=limits)
ctx = ValidationContext(template.profile, caller_limits=limits)
job = create_job(
job_template=template,
job_parameter_values={...},
validation_context=ctx, # optional; defaults to template.default_validation_context()
)
```

A context that strips an extension the template declares raises
`ModelValidationError`:
`create_job requires a context enabling every extension the template declares:
missing EXPR.` An application that does not support an extension rejects the
template at decode, via `supported_extensions`, rather than at job creation.
Deriving the context from `template.profile` — or omitting it — satisfies the
contract by construction.

Because the context contract makes every evaluation error at job creation a
real defect, `create_job` reports them all. A value-dependent failure —
`args: ["{{ 10 // Param.N }}"]` with `N = 0` — fails `create_job` rather than
every session that runs the task. Lowered `max_eval_memory_bytes` /
`max_eval_operations` are enforced inside a conditional whose test only a
worker can resolve, which is the idiomatic construction for one.

Validation outcomes do not depend on the host operating system. Every stage
that evaluates outside host context — template validation and every resolution
`create_job` performs, including its resolved-value re-checks — evaluates
under the POSIX path format, so a PATH value flowing through a `let` binding
into an argument validates identically on Windows and POSIX.

```python
# 5. Bridge to the expression engine.
from openjd.expr import HostContext
expr_profile = profile.to_expr_profile(HostContext.unresolved())
Expand Down
22 changes: 18 additions & 4 deletions src/openjd/_openjd_rs.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -2772,15 +2772,22 @@ class TemplateAction:
@typing.final
class TemplateAmountRequirement:
@property
def name(self) -> builtins.str: ...
def name(self) -> FormatString:
r"""
`@fmtstring` as of openjd-rs#409: a name may carry expressions, and its
§3.3.1.1 constraints are checked on the resolved value. Use `.raw()` for
the template text. The job-side `AmountRequirement.name` is the resolved
`str`.
"""

@property
def min(self) -> typing.Optional[FormatString]: ...
@property
def max(self) -> typing.Optional[FormatString]: ...
def __new__(
cls,
*,
name: builtins.str,
name: FormatString,
min: typing.Optional[FormatString] = None,
max: typing.Optional[FormatString] = None,
) -> TemplateAmountRequirement: ...
Expand All @@ -2790,7 +2797,14 @@ class TemplateAmountRequirement:
@typing.final
class TemplateAttributeRequirement:
@property
def name(self) -> builtins.str: ...
def name(self) -> FormatString:
r"""
`@fmtstring` as of openjd-rs#409: a name may carry expressions, and its
§3.3.2.1 constraints are checked on the resolved value. Use `.raw()` for
the template text. The job-side `AttributeRequirement.name` is the
resolved `str`.
"""

@property
def any_of(self) -> typing.Optional[builtins.list[FormatString]]: ...
@property
Expand All @@ -2802,7 +2816,7 @@ class TemplateAttributeRequirement:
def __new__(
cls,
*,
name: builtins.str,
name: FormatString,
any_of: typing.Optional[typing.Sequence[FormatString]] = None,
all_of: typing.Optional[typing.Sequence[FormatString]] = None,
) -> TemplateAttributeRequirement: ...
Expand Down
Loading
Loading