Repository navigation
fix(workflow): let the later write in an apply batch win #1007
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f1e907f
7a731a5
6c6eaad
625ff68
81d2eb6
4c1402f
50408f2
c1f6eae
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,6 +34,10 @@ | |
| * **Last-writer-wins on widgets** — two concurrent writes to the same widget | ||
| converge on the value with the higher causal ``stamp`` ``[base_version, actor]`` | ||
| (``op_id`` breaks exact ties into a total order), independent of apply order. | ||
| Ops share one stamp within an ``apply_specs`` batch, and ``op_id`` is minted | ||
| in increasing order, so the later write in a batch wins unless a batch guard | ||
| refuses the batch; amendment v1.14 also names a rebuild shape that still | ||
| depends on apply order (#1101). | ||
| The winning stamp per target is tracked in ``_widget_stamps`` (apply-only | ||
| bookkeeping, stripped before serialization). | ||
| * **Deterministic structure** — forking a shared subgraph definition mints an id | ||
|
|
@@ -61,7 +65,9 @@ | |
| import math | ||
| import random | ||
| import re | ||
| import uuid | ||
| import secrets | ||
| import threading | ||
| import time | ||
| from decimal import Decimal | ||
| from typing import Any | ||
|
|
||
|
|
@@ -388,10 +394,25 @@ def mint_id() -> int: | |
| return _ID_FLOOR | random.getrandbits(52) | ||
|
|
||
|
|
||
| _last_op_clock = 0 | ||
| _op_clock_lock = threading.Lock() | ||
|
|
||
|
|
||
| def _new_op_id() -> str: | ||
| """32 lowercase hex chars (§8.2): a nanosecond clock that never repeats or | ||
| goes backwards in this process, then 64 random bits for uniqueness. Every op | ||
| in an ``apply_specs`` batch shares one stamp, so ``op_id`` is what breaks the | ||
| tie; minting it in increasing order makes the later spec win.""" | ||
| global _last_op_clock | ||
| with _op_clock_lock: | ||
| clock = _last_op_clock = max(time.time_ns(), _last_op_clock + 1) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ⚪ Nit — The first half of every emitted
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Documented in 50408f2: the "Scope of the ordering" paragraph now says the prefix carries the minting host's clock to the nanosecond wherever the op_id travels. The ops already carry the actor and base_version, and edit time is not sensitive in this collaboration model, so I kept the full clock rather than coarsening it. |
||
| return f"{clock:016x}{secrets.token_hex(8)}" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ⚪ Nit —
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this. Overflow needs a clock past the year 2554. |
||
|
|
||
|
|
||
| def _new_op(kind: str, actor: str, base_version: int, **fields: Any) -> dict[str, Any]: | ||
| return { | ||
| "op": kind, | ||
| "op_id": uuid.uuid4().hex, | ||
| "op_id": _new_op_id(), | ||
| "actor": actor, | ||
| "base_version": base_version, | ||
| "stamp": [base_version, actor], | ||
|
|
@@ -1313,8 +1334,8 @@ def set_dynamic_group_rows( | |
|
|
||
| The writes run on a copy, so ``workflow`` is unchanged when any is refused. | ||
| The ops share one stamp, so an earlier write to the group at the same | ||
| stamp is refused rather than left to the ``op_id`` tiebreak (revisit once | ||
| #1007 makes the later write in a batch win). ``apply_op`` must replay them | ||
| stamp is refused rather than left to the ``op_id`` tiebreak (kept by | ||
| op-vocabulary amendment v1.14). ``apply_op`` must replay them | ||
| in order, because this workflow model materializes rows from the count. | ||
| """ | ||
| from comfy_cli.cql import engine as _engine | ||
|
|
@@ -3698,8 +3719,7 @@ def unknown_size(class_type: Any) -> list[float] | None: | |
| except KeyError as e: | ||
| raise ValueError(f"spec #{i} ({kind}) is missing required field {e}") from e | ||
| # A later write at the batch's stamp would tie with the replacement | ||
| # and be settled by the random op_id. Revisit once #1007 makes the | ||
| # later write in a batch win. | ||
| # on the op_id tiebreak. Amendment v1.14 keeps this refusal. | ||
| target = _write_target(op) if op.get("op") == "set_widget" else None | ||
| if target in replaced: | ||
| raise ValueError( | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| """Two writes to one register inside an ``apply_specs`` batch: the later spec wins. | ||
|
|
||
| Every op in a batch carries the same stamp ``[base_version, actor]``, so the | ||
| ``op_id`` tiebreak in ``_stamp_key`` decides between them. With uuid4 op_ids the | ||
| earlier write won about half the time and the later one became a silent no-op | ||
| (#1006). Amendment v1.14 mints increasing op_ids, so spec order decides. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import copy | ||
|
|
||
| from comfy_cli import workflow_ops | ||
| from comfy_cli.cql.engine import Graph | ||
|
|
||
| _SUB = {"prompt": ["STRING", {"default": ""}]} | ||
| _MAX_SUB = {**_SUB, "mode": ["COMBO", {"default": "balanced", "options": ["balanced", "quality"]}]} | ||
|
|
||
| GRAPH = Graph.from_object_info( | ||
| { | ||
| "Gen": { | ||
| "input": { | ||
| "required": { | ||
| "model": [ | ||
| "COMFY_DYNAMICCOMBO_V3", | ||
| { | ||
| "options": [ | ||
| {"key": "base", "inputs": {"required": _SUB}}, | ||
| {"key": "max", "inputs": {"required": _MAX_SUB}}, | ||
| ] | ||
| }, | ||
| ], | ||
| "label": ["STRING", {"default": ""}], | ||
| } | ||
| }, | ||
| "input_order": {"required": ["model", "label"]}, | ||
| "output": ["VIDEO"], | ||
| "output_name": ["VIDEO"], | ||
| "category": "test", | ||
| "display_name": "Gen", | ||
| "python_module": "nodes", | ||
| } | ||
| } | ||
| ) | ||
|
|
||
| RUNS = 20 | ||
|
|
||
|
|
||
| def _fresh() -> tuple[dict, int]: | ||
| wf = {"nodes": [], "links": [], "last_node_id": 0, "last_link_id": 0, "version": 0.4} | ||
| wf, op = workflow_ops.add_node(wf, GRAPH, "Gen") | ||
| return wf, op["node_id"] | ||
|
|
||
|
|
||
| def _widget(wf: dict, nid: int, name: str): | ||
| node = next(n for n in wf["nodes"] if n["id"] == nid) | ||
| values = node["widgets_values"] | ||
| return values[workflow_ops._widget_index(GRAPH, "Gen", name, values)] | ||
|
|
||
|
|
||
| def test_later_write_to_the_same_widget_wins(): | ||
| for _ in range(RUNS): | ||
| wf, nid = _fresh() | ||
| wf, _, _ = workflow_ops.apply_specs( | ||
| wf, | ||
| GRAPH, | ||
| [ | ||
| {"op": "set_widget", "node": nid, "widget": "label", "value": "x"}, | ||
| {"op": "set_widget", "node": nid, "widget": "label", "value": "y"}, | ||
| ], | ||
| ) | ||
| assert _widget(wf, nid, "label") == "y" | ||
|
|
||
|
|
||
| def test_later_selector_write_shapes_the_specs_after_it(): | ||
| """A structural register: the second selector write must win, or the | ||
| sub-widget write after it fails with "not found".""" | ||
| for _ in range(RUNS): | ||
| wf, nid = _fresh() | ||
| wf, _, _ = workflow_ops.apply_specs( | ||
| wf, | ||
| GRAPH, | ||
| [ | ||
| {"op": "set_widget", "node": nid, "widget": "model", "value": "base"}, | ||
| {"op": "set_widget", "node": nid, "widget": "model", "value": "max"}, | ||
| {"op": "set_widget", "node": nid, "widget": "model.mode", "value": "quality"}, | ||
| ], | ||
| ) | ||
| assert _widget(wf, nid, "model") == "max" | ||
| assert _widget(wf, nid, "model.mode") == "quality" | ||
|
|
||
|
|
||
| def test_batch_stamps_increase_in_spec_order(): | ||
| wf, nid = _fresh() | ||
| specs = [{"op": "set_widget", "node": nid, "widget": "label", "value": str(i)} for i in range(50)] | ||
| _, ops, _ = workflow_ops.apply_specs(wf, GRAPH, specs) | ||
| keys = [workflow_ops._stamp_key(op) for op in ops] | ||
| assert keys == sorted(keys) and len(set(map(tuple, keys))) == len(keys) | ||
|
|
||
|
|
||
| def test_batch_replays_to_the_same_graph_in_any_order_and_twice(): | ||
| base, nid = _fresh() | ||
| specs = [ | ||
| {"op": "set_widget", "node": nid, "widget": "label", "value": "x"}, | ||
| {"op": "set_widget", "node": nid, "widget": "label", "value": "y"}, | ||
| ] | ||
| local, ops, _ = workflow_ops.apply_specs(copy.deepcopy(base), GRAPH, specs) | ||
| expected = workflow_ops.canonical(local) | ||
| for order in (ops, ops[::-1], ops + ops[::-1]): | ||
| wf = copy.deepcopy(base) | ||
| for op in order: | ||
| workflow_ops.apply_op(wf, op, GRAPH) | ||
| assert workflow_ops.canonical(wf) == expected | ||
|
|
||
|
|
||
| def test_selector_batch_replays_to_the_same_graph_in_any_order_and_twice(): | ||
| base, nid = _fresh() | ||
| specs = [ | ||
| {"op": "set_widget", "node": nid, "widget": "model", "value": "base"}, | ||
| {"op": "set_widget", "node": nid, "widget": "model", "value": "max"}, | ||
| {"op": "set_widget", "node": nid, "widget": "model.mode", "value": "quality"}, | ||
| ] | ||
| local, ops, _ = workflow_ops.apply_specs(copy.deepcopy(base), GRAPH, specs) | ||
| expected = workflow_ops.canonical(local) | ||
| for order in (ops, ops[::-1], ops + ops[::-1]): | ||
| wf = copy.deepcopy(base) | ||
| for op in order: | ||
| workflow_ops.apply_op(wf, op, GRAPH) | ||
| assert workflow_ops.canonical(wf) == expected |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟢 Low — This module-level
threading.Lockisn't reinitialized afterfork(). If a multithreaded host forks while another thread is inside_new_op_id's critical section, the child inherits a held lock that no surviving thread owns, and its first mint hangs forever. The window is narrow, but the fix is cheap: reset the lock withos.register_at_fork(after_in_child=...). Raised by 2 of 6 reviewers (gpt-5.6-sol-xhigh adversarial, gpt-5.6-sol-xhigh edge-case).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not changing this. comfy-cli mints ops on one thread and does not fork while minting. The lock covers two integer operations. Adding an at-fork reset would guard a host this code does not run in.