diff --git a/README.md b/README.md index 07b6363..e4e6645 100644 --- a/README.md +++ b/README.md @@ -70,13 +70,15 @@ as shown below). This will also produce a `.template.py` file in the implementing components in python. Every port, command, event, telemetry channel, parameter and internal port of the component is bound, as is the -overflow hook of each member with `hook` queue-full behavior (see below). Three component features cannot be bound -yet, because F Prime declares a handler for each that has no Python equivalent: +overflow hook of each member with `hook` queue-full behavior and each instance of an internal state machine (see +below). A few component features cannot be bound yet, because F Prime declares a handler for each that has no Python +equivalent: | Feature | Why | What happens | | --- | --- | --- | | serial port | its handler takes a serialization buffer | rejected with an error naming the port | -| state machine instance | its actions and guards take a state machine id and a signal | rejected with an error naming the component | +| instance of an external state machine | its actions and guards are declared by hand-written C++ (`_Interface`), not in the model | rejected with an error naming the instance | +| state machine instance with `hook` queue-full behavior | its overflow hook takes the queued signal as a raw serial buffer | rejected with an error naming the instance | | data product container | its handler takes a `Fw::DpContainer` | rejected with an error naming the component | In each case the autocoder refuses rather than generating a class that would leave the handler unimplemented and @@ -116,6 +118,73 @@ def DO_cmdOverflowHook(self, opCode, cmdSeq): An exception raised in a hook is not caught by the generated C++, as for any handler (see [Strings](#strings)), but here it reaches the caller's thread. +### State Machines + +A component may instantiate internal state machines: ones whose actions, guards, signals and states are defined in +FPP. Take this one, instantiated twice: + +``` +module Ref { + state machine Blinker { + action store: U32 + guard allow: U32 + signal advance: U32 + signal back + initial enter IDLE + state IDLE { on advance if allow do { store } enter BUSY } + state BUSY { on back enter IDLE } + } + + @ fprime-python + queued component PySm { + ... + state machine instance first: Blinker + state machine instance second: Blinker + } +} +``` + +Each action and guard is a Python handler, named as F Prime names it after the machine's flattened qualified name: +`Ref_Blinker_action_store(self, smId, signal, value)` and `Ref_Blinker_guard_allow(self, smId, signal, value)` for a +machine `Ref.Blinker`. Every handler takes `smId` and `signal`; one whose action or guard declares a type also takes +the signal's `value`. The handlers belong to the machine, not to an instance, so `first` and `second` call the same +ones, and `smId` says which: it is an `enum.Enum` bound inside the component's class, `fprime_py.Ref.PySm.SmId`, with +one value per instance named after it (`smId.name == "first"`). A guard is `const` in C++ and must return a `bool`; +anything else, `None` included, raises `TypeError`. The template's guard stubs return `False`, which leaves the guarded +transition untaken. Every action and guard must be implemented: a missing one raises `AttributeError` from the +generated `init`. + +Python drives each instance through the methods F Prime generates for it: + +- `first_sendSignal_advance(value)` and `first_sendSignal_back()` queue a signal on the component's queue. A string + value is checked first against the size F Prime serializes it at: its declared size, else + `FW_FIXED_LENGTH_STRING_SIZE`. On a full queue the instance's queue-full behavior applies: `assert` (the default) + asserts, `drop` drops the signal, and `block` waits for room with the GIL released, so the dispatching thread can run + the handlers that make it. Never send to a `block` instance from the thread that dispatches its queue (any action or + guard, or on a queued component the code that calls `doDispatch`): on a full queue that send waits forever. +- `first_getState()` returns the current state as a `fprime_py.Ref.Blinker_State`, an F Prime enumeration whose `e` + is a `Blinker_State.T`, e.g. `Blinker_State.T.IDLE`. + +The machine's states and signals are bound once, into the Python module of the FPP module defining the machine, as +`Blinker_State`, an F Prime enumeration class whose values are in `Blinker_State.T`, and `Blinker_Signal`, an +`enum.Enum` of the values `signal` takes, in the order of their numbers. A machine defined inside a component +is named as F Prime names it, with the component folded in: `PySm_Local_State` and `PySm_Local_Signal` in the +component's module. Both enumerations include F Prime's internal entries, because Python can see them: +`Blinker_State.T.__FPRIME_UNINITIALIZED` is the state before the machine starts, and an action on the initial +transition receives `Blinker_Signal.__FPRIME_INITIAL_TRANSITION`. Inside a class body Python mangles such names, so +spell them `Blinker_Signal["__FPRIME_INITIAL_TRANSITION"]` there. + +- **Thread.** Actions and guards run on the thread dispatching the component's queue, holding the GIL: the + component's own thread when it is active, and whatever calls `doDispatch` or `dispatchAvailableMessages` when it is + queued. `sendSignal` only queues the signal, and the transition happens when the queue is next dispatched. +- **Initial transition.** The actions and guards of the initial transition run during the component's `init`, on the + thread setting up the topology, before the component's queue is created and before its ports are connected. They + must not send signals or invoke ports; they may only set up Python state. +- **Exceptions.** An exception raised in an action or guard is not caught by the generated C++, as for any handler + (see [Strings](#strings)). One raised on the initial transition propagates out of the component's `init`. +- **Instances defined elsewhere.** A machine defined inside another component is bound only with that component, so + it is rejected; define it in a module instead. + ## Setting a Deployment Up With Python Bindings The deployment topology that instantiates Python components must also carry the `@ fprime-python` annotation. Only an diff --git a/src/fprime_python/autocode/binding_generator.py b/src/fprime_python/autocode/binding_generator.py index 1f3e0e9..43b8666 100644 --- a/src/fprime_python/autocode/binding_generator.py +++ b/src/fprime_python/autocode/binding_generator.py @@ -28,7 +28,7 @@ #: Doc comment placed on every generated initialization function INIT_FUNCTION_COMMENT = """\\brief bind {fqn} into Python -This function initializes the Python bindings for the FPP type {fqn}. It should be called +This function initializes the Python bindings for {subject}. It should be called from within the pybind11 module initialization macro at top level. One language to rule them all, one language to find them... @@ -113,6 +113,16 @@ def cpp_fqn(self) -> str: """ The definition's C++-qualified name """ return cpp_name(self.fpp_name) + @property + def doc_name(self) -> str: + """ The name the generated initialization function's comment says it binds """ + return self.cpp_fqn + + @property + def doc_subject(self) -> str: + """ What the generated initialization function's comment says it binds, in a sentence """ + return f"the FPP type {self.cpp_fqn}" + @property def init_function_name(self) -> str: """ The name of the generated initialization function @@ -187,11 +197,16 @@ def document(self) -> CppDocBuilder: function = doc.function( self.init_function_name, params=[("pybind11::module_&", "m")], - comment=lines(INIT_FUNCTION_COMMENT.format(fqn=self.cpp_fqn), margin=None), + comment=lines(INIT_FUNCTION_COMMENT.format(fqn=self.doc_name, subject=self.doc_subject), margin=None), ) self.bind(function.body) return doc + @property + def scope(self) -> str: + """ The FPP-qualified name of the scope whose Python submodule the definition is bound into """ + return scope_of(self.fpp_name) + def invocation(self) -> Dict[str, str]: """ The call the module initialization has to make to bind this definition @@ -199,7 +214,7 @@ def invocation(self) -> Dict[str, str]: A mapping of the FPP scope the definition lives in to the statement invoking its initialization function against that scope's pybind11 submodule """ - scope = scope_of(self.fpp_name) + scope = self.scope submodule_variable = f"{MODULE_VARIABLE_PREFIX}{flat_name(scope)}" return {scope: f"(void) {self.init_function_name}({submodule_variable});"} diff --git a/src/fprime_python/autocode/component_generator.py b/src/fprime_python/autocode/component_generator.py index ee63270..d7a0e11 100644 --- a/src/fprime_python/autocode/component_generator.py +++ b/src/fprime_python/autocode/component_generator.py @@ -19,9 +19,9 @@ from .constants import SELF_MEMBER, SUPPORT_HEADER, TOOL_NAME from .include import IncludeManager from .view import ( + ArgumentView, CommandView, ComponentView, - FormalParameterView, HandlerView, OverflowHookView, ParameterView, @@ -55,8 +55,9 @@ def checked_def( name: str, fqn: str, prefix: Sequence[Tuple[str, str, str]], - params: Sequence[FormalParameterView], + params: Sequence[ArgumentView], cap: Optional[str] = None, + release_gil: bool = False, ) -> Optional[str]: """ Get a binding that checks each sized string argument against the size it is serialized at before the call @@ -71,13 +72,14 @@ def checked_def( prefix: The method's leading parameters that are not modeled, as (C++ type, name, doc) triples params: The method's modeled parameters, in call order cap: The configuration constant that also bounds the method's strings, None when only the declared size does + release_gil: Whether the call releases the GIL once the strings are checked, as a call that may block must Returns: The pybind11 `.def(...)` call, or None when no argument needs a check """ sizes = { param.name: size for param in params - if (size := param.string_size if cap is not None else param.declared_size) is not None + if (size := param.string_size if cap is not None else param.checked_size) is not None } sized = [param for param in params if param.name in sizes] if not sized: @@ -92,6 +94,7 @@ def checked_def( return ( f'.def("{name}", []({fqn}& self, {", ".join(declarations)}) {{\n' f"{checks}" + f"{' pybind11::gil_scoped_release released{};' + chr(10) if release_gil else ''}" f' return self.{name}({", ".join(arguments)});\n' "})" ) @@ -103,7 +106,7 @@ def base_class_methods(component: ComponentView) -> List[str]: F Prime declares these `protected`, so the generated class re-exposes them with using-declarations and the bindings take their addresses. They are the methods a Python implementation calls to act on the rest of the topology: dispatching its own queue, reading the time, responding to commands, - invoking its output ports, and sending events. + invoking its output ports, sending events, and reading and signalling its state machine instances. Args: component: The component to list the methods of @@ -123,6 +126,9 @@ def base_class_methods(component: ComponentView) -> List[str]: methods += [port.connection_check_name for port in component.output_ports] methods += [port.invoker_name for port in component.internal_ports] methods += [event.dispatch_name for event in component.events] + for instance in component.state_machine_instances: + methods.append(instance.get_state_name) + methods += [name for name, _ in instance.send_signals] return methods @@ -134,32 +140,50 @@ class ComponentBindingGenerator(BindingGenerator): """ def __init__( - self, include_manager: IncludeManager, symbol: fpp.Symbol.Variant, component: fpp.Component + self, + include_manager: IncludeManager, + analysis: fpp.Analysis, + symbol: fpp.Symbol.Variant, + component: fpp.Component, ) -> None: """ Initialize the generator for one component Args: include_manager: Resolves the include paths of the headers the binding needs + analysis: The analyzed model the component is part of symbol: The symbol of the component being bound component: The analyzed component being bound """ super().__init__(include_manager, symbol) - self.component = ComponentView(component) + self.component = ComponentView(analysis, component) def bind(self, body: Body) -> None: """ Write the pybind11 statements binding this component """ fqn = self.cpp_fqn component = self.component - signatures: Dict[str, Tuple[Sequence[Tuple[str, str, str]], List[FormalParameterView], Optional[str]]] = {} + signatures: Dict[str, Tuple[Sequence[Tuple[str, str, str]], Sequence[ArgumentView], Optional[str]]] = {} for port in component.output_ports: signatures[port.invoker_name] = (port.HANDLER_PREFIX, port.parameters, None) for internal in component.internal_ports: signatures[internal.invoker_name] = (internal.HANDLER_PREFIX, internal.parameters, None) for event in component.events: signatures[event.dispatch_name] = ((), event.parameters, "FW_LOG_STRING_MAX_SIZE") + # A signal sent to a full queue that blocks waits holding no GIL, so the dispatching thread can run + # the actions and guards that make room + blocking = set() + for instance in component.state_machine_instances: + for name, values in instance.send_signals: + signatures[name] = ((), values, None) + if instance.blocks_when_full: + blocking.add(name) links = [] for method in base_class_methods(component): - checked = checked_def(method, fqn, *signatures[method]) if method in signatures else None + release_gil = method in blocking + checked = ( + checked_def(method, fqn, *signatures[method], release_gil=release_gil) if method in signatures else None + ) + if checked is None and release_gil: + checked = f'.def("{method}", &{fqn}::{method}, pybind11::call_guard())' links.append(checked if checked is not None else standard_def(method, fqn)) # A parameter is read through the generated helper rather than through F Prime's own getter, # because the getter reports validity through an out parameter that has no Python equivalent @@ -175,7 +199,25 @@ def bind(self, body: Body) -> None: ) for channel in self.component.channels ] - body.raw(expression_chain(f'pybind11::class_<{fqn}>(m, "{self.name}")', links)) + instances = component.state_machine_instances + if not instances: + body.raw(expression_chain(f'pybind11::class_<{fqn}>(m, "{self.name}")', links)) + return + # The state machine ids are bound inside the component's class, so the class is kept to bind them into + body.lines(f'pybind11::class_<{fqn}> component(m, "{self.name}");', margin=None) + body.raw(expression_chain("component", links)) + body.blank() + body.raw( + expression_chain( + f'pybind11::native_enum<{fqn}::SmId>(component, "SmId", "enum.Enum")', + [f'.value("{instance.name}", {fqn}::SmId::{instance.name})' for instance in instances] + + [".finalize()"], + ) + ) + + def cpp_system_includes(self) -> List[str]: + """ System headers the generated source needs: `native_enum` for the state machine ids """ + return ["pybind11/native_enum.h"] if self.component.state_machine_instances else [] def cpp_includes(self) -> List[str]: """ Get the C++ includes for a component type @@ -284,6 +326,14 @@ class {name}({name}Base): pass ''' +#: Stub for one state machine guard in the Python implementation template. A guard has to return a bool, and +#: False is the safe default: the guarded transition is not taken until the guard is implemented. +PYTHON_GUARD_HANDLER_TEMPLATE = ''' def {handler}(self{args}): + """ Handle the {name} {kind} """ + # TODO: Implement {kind}, returning a bool. Returning False (this stub) blocks the transition + return False +''' + #: Stub for one command handler or command overflow hook in the Python implementation template PYTHON_COMMAND_HANDLER_TEMPLATE = ''' def {handler}(self{args}): """ Handle the {name} {kind} """ @@ -304,18 +354,23 @@ class ComponentImplementationGenerator(object): """ def __init__( - self, include_manager: IncludeManager, symbol: fpp.Symbol.Variant, component: fpp.Component + self, + include_manager: IncludeManager, + analysis: fpp.Analysis, + symbol: fpp.Symbol.Variant, + component: fpp.Component, ) -> None: """ Initialize the generator for one component Args: include_manager: Resolves the include paths of the headers the implementation needs + analysis: The analyzed model the component is part of symbol: The symbol of the component being implemented component: The analyzed component being implemented """ self.include_manager = include_manager self.symbol = symbol - self.component = ComponentView(component) + self.component = ComponentView(analysis, component) @property def name(self) -> str: @@ -363,6 +418,14 @@ def _write_init(self, body: Body) -> None: f'throw pybind11::attribute_error("{self.name} does not implement overflow hook' f' {hook.handler_name}");' ) + # Likewise an action or guard runs only on the transitions that name it, which may be rare + for handler in self.component.state_machine_handlers: + with body.if_(f'!pybind11::hasattr(pybind11::type::of(this->{SELF_MEMBER}), "{handler.handler_name}")'): + body.line(f"this->{SELF_MEMBER} = pybind11::none();") + body.line( + f'throw pybind11::attribute_error("{self.name} does not implement {handler.HANDLER_KIND}' + f' {handler.handler_name}");' + ) body.comment("Continue the standard initialization of F Prime") arguments = ", ".join(name for _, name, _ in self._init_params()) body.line(f"{self.component.base_class}::init({arguments});") @@ -395,6 +458,13 @@ def _write_forwarding_call(self, body: Body, handler: HandlerView) -> None: body.line(f"{call};") return body.line(f"pybind11::object return_value = {call};") + if handler.RETURNS_STRICT_BOOL: + # pybind11 would convert any object to a bool, None included, so a guard that forgot to return + # would silently block its transition + with body.if_("!pybind11::isinstance(return_value)"): + body.line( + f'throw pybind11::type_error("{self.name}.{handler.handler_name} must return a bool");' + ) size = handler.return_declared_size if size is None: body.line(f"return return_value.cast<{handler.handler_return_type}>();") @@ -452,6 +522,7 @@ def _write_handlers(self, cls: ClassBuilder) -> None: handler.handler_name, ret=handler.handler_return_type, params=handler.handler_parameters, + const=handler.IS_CONST, override=True, comment=handler.description, ) @@ -470,6 +541,9 @@ def _write_exposed_members(self, cls: ClassBuilder) -> None: using_statements = base_class_methods(component) + [ channel.write_name for channel in component.channels ] + # The state machine ids name the instances to Python, which compares an action's `smId` against them + if component.state_machine_instances: + using_statements.append("SmId") if using_statements: with cls.public("Base class members exposed to Python"): # The method names come from the model, so this is not margin-stripped @@ -587,6 +661,13 @@ def python_implementation(self) -> str: ) ) continue + if handler.RETURNS_STRICT_BOOL: + stubs.append( + PYTHON_GUARD_HANDLER_TEMPLATE.format( + handler=handler.handler_name, name=handler.name, kind=handler.HANDLER_KIND, args=args + ) + ) + continue return_type = handler.handler_return_type stubs.append( PYTHON_PORT_HANDLER_TEMPLATE.format( diff --git a/src/fprime_python/autocode/cpp_types.py b/src/fprime_python/autocode/cpp_types.py index c6ab29a..4bc205d 100644 --- a/src/fprime_python/autocode/cpp_types.py +++ b/src/fprime_python/autocode/cpp_types.py @@ -91,6 +91,23 @@ def value_type(type_: Optional[fpp.Type.Variant], string_class: StringClass) -> return type_name(type_, string_class) +def value_parameter_type(type_: fpp.Type.Variant, string_class: StringClass) -> str: + """ The C++ spelling of a type where it is passed in, not by `ref`: a primitive by value, else by `const` reference + + Args: + type_: The FPP semantic type to spell + string_class: The string class to use should the type be a modeled string + Returns: + The C++ parameter type, e.g. "U32", "const Ref::PySimple&" + """ + name = type_name(type_, string_class) + # An alias of a primitive is passed by value under the alias's own name, which is how F Prime's + # autocoder spells it. `is_primitive` does not look through aliases, so the aliased type is checked. + if type_.underlying_type.is_primitive: + return name + return f"const {name}&" + + def formal_parameter_type(param: fpp.ast.FormalParam, string_class: StringClass) -> str: """ The C++ spelling of a formal parameter, reference qualifiers included @@ -105,11 +122,7 @@ def formal_parameter_type(param: fpp.ast.FormalParam, string_class: StringClass) name = type_name(type_, string_class) if param.kind == fpp.ast.FormalParamKind.Ref: return f"{name}&" - # An alias of a primitive is passed by value under the alias's own name, which is how F Prime's - # autocoder spells it. `is_primitive` does not look through aliases, so the aliased type is checked. - if type_.underlying_type.is_primitive: - return name - return f"const {name}&" + return value_parameter_type(type_, string_class) def struct_member_getter(type_: fpp.Type.Variant) -> Tuple[str, bool]: diff --git a/src/fprime_python/autocode/include.py b/src/fprime_python/autocode/include.py index ba9d690..f10959e 100644 --- a/src/fprime_python/autocode/include.py +++ b/src/fprime_python/autocode/include.py @@ -36,6 +36,7 @@ class IncludeManager(object): fpp.Symbol.Constant: "Constant", fpp.Symbol.EnumType: "Enum", fpp.Symbol.Port: "Port", + fpp.Symbol.StateMachine: "StateMachine", fpp.Symbol.StructType: "Serializable", fpp.Symbol.Topology: "Topology", } diff --git a/src/fprime_python/autocode/types_generator.py b/src/fprime_python/autocode/types_generator.py index 4fa8f89..3c5c11f 100644 --- a/src/fprime_python/autocode/types_generator.py +++ b/src/fprime_python/autocode/types_generator.py @@ -13,6 +13,8 @@ from .binding_generator import BindingGenerator, expression_chain from .cpp_types import StringClass, struct_member_getter, type_name from .include import IncludeManager +from .names import flat_name +from .view import StateMachineView class ArrayBindingGenerator(BindingGenerator): @@ -159,31 +161,136 @@ def __init__( def bind(self, body: Body) -> None: """ Write the pybind11 statements binding this enumeration """ - fqn = self.cpp_fqn - # The C++ name comes from the model, so this is not margin-stripped - body.lines( - f'pybind11::class_<{fqn}> enumeration(m, "{self.name}");\n' - f'enumeration.def_readwrite("e", &{fqn}::e);', - margin=None, + write_enum_wrapper(body, self.cpp_fqn, self.name, [constant.name for constant in self.enum_type.node.constants]) + + def cpp_system_includes(self) -> List[str]: + """ Get any system includes required by this type generator """ + return ["pybind11/native_enum.h"] + + +def write_enum_wrapper(body: Body, fqn: str, python_name: str, constants: List[str]) -> None: + """ Write the pybind11 statements binding one F Prime enumeration class and its nested `T` + + Args: + body: The body of the initialization function to write the statements into + fqn: The C++-qualified name of the enumeration class + python_name: The name the class is bound into the module `m` under + constants: The names of the enumeration's constants + """ + # The C++ name comes from the model, so this is not margin-stripped + body.lines( + f'pybind11::class_<{fqn}> enumeration(m, "{python_name}");\n' + f'enumeration.def_readwrite("e", &{fqn}::e);', + margin=None, + ) + body.blank() + # `native_enum` has to be finalized before anything else touches the enclosing scope, so the + # value chain is emitted as one statement of its own. + body.raw( + expression_chain( + f'pybind11::native_enum<{fqn}::T>(enumeration, "T", "enum.Enum")', + [f'.value("{constant}", {fqn}::{constant})' for constant in constants] + + [".export_values()", ".finalize()"], ) + ) + body.blank() + # Constructing the wrapper from one of its own values is the natural Python spelling, and can + # only be bound once T exists. + body.lines(f"enumeration.def(pybind11::init<{fqn}::T>());", margin=None) + + +class StateMachineBindingGenerator(BindingGenerator): + """ Generator for the bindings of an internal state machine's states and signals into Python + + F Prime generates the states as an ordinary enumeration class, `_State`, which is bound the way + any other enumeration is, and the signals as an `enum class Signal` nested in `StateMachineBase`, + which is bound as a `pybind11::native_enum` named `_Signal` beside it. A machine is bound once, + by the module defining it, however many components instantiate it: pybind11 refuses to register a type + twice. + + Example Output: + ```cpp + pybind11::class_ enumeration(m, "Blinker_State"); + ... + pybind11::native_enum(m, "Blinker_Signal", "enum.Enum") + .value("__FPRIME_INITIAL_TRANSITION", Ref::BlinkerStateMachineBase::Signal::__FPRIME_INITIAL_TRANSITION) + .value("advance", Ref::BlinkerStateMachineBase::Signal::advance) + .finalize(); + ``` + """ + + FILE_BASE_SUFFIX = "StateMachineBindingAc" + INVOCATION_SUFFIX = "StateMachineBinding" + + def __init__(self, include_manager: IncludeManager, machine: StateMachineView) -> None: + """ Initialize the generator for one internal state machine + + Args: + include_manager: Resolves the include paths of the headers the binding needs + machine: The state machine being bound + """ + super().__init__(include_manager, machine.symbol) + self.machine = machine + + @property + def name(self) -> str: + """ The state machine's name within its module, which F Prime names its files after """ + return self.machine.base_name + + @property + def cpp_fqn(self) -> str: + """ The C++-qualified name of the state machine base class """ + return self.machine.base_class + + @property + def doc_name(self) -> str: + """ The bindings are of the machine's states and signals rather than of its base class """ + return f"the states and signals of {self.machine.fpp_name}" + + @property + def doc_subject(self) -> str: + """ The bindings are of the machine's states and signals rather than of its base class """ + return f"the states and signals of the FPP state machine {self.machine.fpp_name}" + + @property + def scope(self) -> str: + """ The module the state machine's C++ names live in, even when it is defined inside a component """ + return self.machine.scope + + @property + def init_function_name(self) -> str: + """ The initialization function's name, kept apart from that of a type or component of the same name """ + return f"init_{flat_name(self.fpp_name)}_StateMachine" + + @property + def include_guard(self) -> str: + """ The include guard of the generated header """ + return f"FPRIME_PYTHON_{flat_name(self.fpp_name)}_STATE_MACHINE_HPP" + + def cpp_includes(self) -> List[str]: + """ The generated header, and the state machine header F Prime generates, which includes its states """ + return [ + self.include_manager.get_sibling_path(self.symbol, f"{self.file_base}.hpp"), + self.include_manager.get_sibling_path(self.symbol, self.machine.header_name), + ] + + def bind(self, body: Body) -> None: + """ Write the pybind11 statements binding the state machine's states and signals """ + machine = self.machine + write_enum_wrapper(body, machine.state_type, machine.state_python_name, machine.state_constants) body.blank() - # `native_enum` has to be finalized before anything else touches the enclosing scope, so the - # value chain is emitted as one statement of its own. - constants = [constant.name for constant in self.enum_type.node.constants] + # Bound straight into the module, so its values are not exported: they would land beside every other + # definition of the module, where the signals of two machines could collide body.raw( expression_chain( - f'pybind11::native_enum<{fqn}::T>(enumeration, "T", "enum.Enum")', - [f'.value("{constant}", {fqn}::{constant})' for constant in constants] - + [".export_values()", ".finalize()"], + f'pybind11::native_enum<{machine.signal_type}>(m, "{machine.signal_python_name}", "enum.Enum")', + [f'.value("{signal}", {machine.signal_type}::{signal})' for signal in machine.signal_constants] + + [".finalize()"], ) ) - body.blank() - # Constructing the wrapper from one of its own values is the natural Python spelling, and can - # only be bound once T exists. - body.lines(f"enumeration.def(pybind11::init<{fqn}::T>());", margin=None) def cpp_system_includes(self) -> List[str]: - """ Get any system includes required by this type generator """ + """ Get any system includes required by this generator """ return ["pybind11/native_enum.h"] diff --git a/src/fprime_python/autocode/view.py b/src/fprime_python/autocode/view.py index 2733e13..c6ce7af 100644 --- a/src/fprime_python/autocode/view.py +++ b/src/fprime_python/autocode/view.py @@ -11,15 +11,19 @@ from __future__ import annotations from abc import ABC, abstractmethod -from typing import List, Optional, Sequence, Tuple +from functools import cached_property +from typing import List, Optional, Sequence, Tuple, Union import fpp from .constants import FPRIME_PYTHON_ANNOTATION from .cpp_types import StringClass, formal_parameter_type, type_name, value_type -from .names import cpp_name, namespaces_of +from .names import cpp_name, flat_name, namespaces_of, scope_of +#: The signal F Prime passes to an action on a state machine's initial transition +INITIAL_TRANSITION_SIGNAL = "__FPRIME_INITIAL_TRANSITION" + #: The C++ token F Prime builds a `log_` method name from, per FPP event severity EVENT_SEVERITY_TOKENS = { fpp.ast.EventSeverity.ActivityHigh: "ACTIVITY_HI", @@ -75,7 +79,45 @@ def string_size_expression(type_: Optional[fpp.Type.Variant]) -> Optional[str]: return "FW_FIXED_LENGTH_STRING_SIZE" if size is None else str(size) -class FormalParameterView: +class ArgumentView(ABC): + """ One argument of a handler or of a method bound into Python, with what the bindings need to know of it """ + + @property + @abstractmethod + def name(self) -> str: + """ The argument's name """ + + @property + @abstractmethod + def cpp_type(self) -> str: + """ The argument's C++ type, reference qualifiers included """ + + @property + @abstractmethod + def resolved_type(self) -> fpp.Type.Variant: + """ The argument's FPP semantic type """ + + @property + def declared_size(self) -> Optional[int]: + """ The argument's declared `string size`, None unless it is a string of explicit size """ + return declared_string_size(self.resolved_type) + + @property + def string_size(self) -> Optional[str]: + """ The C++ expression for the size of the argument's string, None unless it is a string """ + return string_size_expression(self.resolved_type) + + @property + def checked_size(self) -> Optional[str]: + """ The size a bound method checks the argument's string against, None when it is not checked + + Only a declared size is checked by default: a string of default size is bounded by its capacity alone. + """ + size = self.declared_size + return None if size is None else str(size) + + +class FormalParameterView(ArgumentView): """ One formal parameter of a port, command or event """ def __init__(self, param: fpp.ast.FormalParam, string_class: StringClass) -> None: @@ -94,14 +136,59 @@ def cpp_type(self) -> str: return formal_parameter_type(self.param, self.string_class) @property - def declared_size(self) -> Optional[int]: - """ The parameter's declared `string size`, None unless it is a string of explicit size """ - return declared_string_size(self.param.type_name.resolved_type) + def resolved_type(self) -> fpp.Type.Variant: + """ The parameter's FPP semantic type """ + type_ = self.param.type_name.resolved_type + assert type_ is not None + return type_ + + +class StateMachineValueView(ArgumentView): + """ The value a state machine signal carries, and that the actions and guards it reaches take + + F Prime names it `value` everywhere and passes it as a port passes an argument: a primitive by value, + anything else by `const` reference, a string as `Fw::StringBase`. + """ + + def __init__(self, type_name: fpp.ast.TypeName) -> None: + """ Wrap the type an action, guard or signal declares """ + self.type_name = type_name @property - def string_size(self) -> Optional[str]: - """ The C++ expression for the size of the parameter's string, None unless it is a string """ - return string_size_expression(self.param.type_name.resolved_type) + def name(self) -> str: + """ The name F Prime gives the value """ + return "value" + + @property + def cpp_type(self) -> str: + """ The value's C++ type, reference qualifiers included + + Unlike a port argument, a value of an alias type is passed by `const` reference even when the alias names a + primitive: F Prime passes a value by value only when its declared type is itself primitive. + """ + type_ = self.resolved_type + name = type_name(type_, StringClass.PORT) + return name if type_.is_primitive else f"const {name}&" + + @property + def checked_size(self) -> Optional[str]: + """ The size F Prime serializes the value's string at: its declared size, else `FW_FIXED_LENGTH_STRING_SIZE` + + F Prime truncates a longer string where it queues the signal, so the binding rejects it first. + """ + return self.string_size + + @property + def resolved_type(self) -> fpp.Type.Variant: + """ The value's FPP semantic type """ + type_ = self.type_name.resolved_type + assert type_ is not None + return type_ + + +def state_machine_values(type_name: Optional[fpp.ast.TypeName]) -> List[ArgumentView]: + """ The value arguments of a state machine action, guard or signal: one when it declares a type, else none """ + return [] if type_name is None else [StateMachineValueView(type_name)] class UnsupportedModelError(Exception): @@ -124,6 +211,12 @@ class HandlerView(ABC): #: What the handler handles, as it reads in a generated comment HANDLER_KIND: str = "" + #: Whether F Prime declares the handler `const`, as it does a state machine guard + IS_CONST: bool = False + + #: Whether the handler answers with a `bool` that Python must return as such, as a state machine guard does + RETURNS_STRICT_BOOL: bool = False + @property @abstractmethod def name(self) -> str: @@ -141,12 +234,12 @@ def description(self) -> str: @property @abstractmethod - def parameters(self) -> List[FormalParameterView]: + def parameters(self) -> Sequence[ArgumentView]: """ The member's own formal parameters, in declaration order """ @property def handler_return_type(self) -> str: - """ The C++ return type of the handler; only a port can return a value """ + """ The C++ return type of the handler; only a port or a state machine guard can return a value """ return "void" @property @@ -353,11 +446,255 @@ def handler_name(self) -> str: return f"{self.owner.name}_{suffix}" @property - def parameters(self) -> List[FormalParameterView]: + def parameters(self) -> Sequence[ArgumentView]: """ The owning member's formal parameters, none for a command """ return [] if self.is_command_hook else self.owner.parameters +class StateMachineView: + """ One state machine definition, with the C++ and Python names F Prime and the bindings give it + + F Prime generates a state machine into the namespace of its enclosing modules even when it is defined + inside a component: the component's name is folded into the machine's instead, so `M` in component `C` + of module `Ref` becomes `Ref::C_MStateMachineBase` with its states in `Ref::C_M_State`. The Python + bindings follow the same names, under the module's Python scope. + """ + + def __init__(self, analysis: fpp.Analysis, symbol: fpp.Symbol.Variant) -> None: + """ Wrap the state machine a symbol defines + + Args: + analysis: The analyzed model the state machine is part of + symbol: The symbol of the state machine definition + """ + self.analysis = analysis + self.symbol = symbol + self.state_machine = analysis.state_machine_map[symbol] + + @property + def fpp_name(self) -> str: + """ The state machine's FPP-qualified name """ + return self.symbol.qualified_name + + @property + def is_internal(self) -> bool: + """ Whether the model defines the state machine's behavior, rather than hand-written C++ """ + return self.state_machine.kind == fpp.Kind.Internal + + @property + def enclosing_component(self) -> Optional[fpp.Symbol.Variant]: + """ The component the state machine is defined inside, None when it is defined in a module """ + parent = self.analysis.parent_symbol_map.get(self.symbol) + return parent if isinstance(parent, fpp.Symbol.Component) else None + + @property + def scope(self) -> str: + """ The FPP-qualified name of the module the state machine's C++ and Python names live in """ + component = self.enclosing_component + return scope_of(self.fpp_name if component is None else component.qualified_name) + + @property + def base_name(self) -> str: + """ The state machine's name within its module, with any enclosing component's folded in """ + component = self.enclosing_component + name = self.symbol.unqualified_name + return name if component is None else f"{component.unqualified_name}_{name}" + + def _in_scope(self, name: str) -> str: + """ The C++-qualified spelling of a name in the state machine's module """ + return cpp_name(f"{self.scope}.{name}") if self.scope else name + + @property + def base_class(self) -> str: + """ The C++-qualified name of the state machine base class F Prime generates """ + return self._in_scope(f"{self.base_name}StateMachineBase") + + @property + def signal_type(self) -> str: + """ The C++-qualified name of the state machine's signal enumeration """ + return f"{self.base_class}::Signal" + + @property + def state_type(self) -> str: + """ The C++-qualified name of the F Prime enumeration of the state machine's states """ + return self._in_scope(f"{self.base_name}_State") + + @property + def header_name(self) -> str: + """ The file name of the header F Prime generates for the state machine """ + return f"{self.base_name}StateMachineAc.hpp" + + @property + def interface_name(self) -> str: + """ The C++-qualified name of the interface an external state machine's C++ implementation provides """ + return self._in_scope(f"{self.base_name}_Interface") + + @property + def nested_class_name(self) -> str: + """ The name of the class F Prime nests in a component's base class for this state machine + + It is the machine's FPP-qualified name, flattened, and prefixes the name of every action and guard + handler. + """ + return flat_name(self.fpp_name) + + @property + def state_python_name(self) -> str: + """ The name the state enumeration is bound into Python under, its C++ name in the module """ + return f"{self.base_name}_State" + + @property + def signal_python_name(self) -> str: + """ The name the signal enumeration is bound into Python under, alongside the state enumeration """ + return f"{self.base_name}_Signal" + + @property + def state_constants(self) -> List[str]: + """ The constants of the state enumeration fpp synthesizes, including its uninitialized state """ + for member in self.state_machine.node.members or []: + if isinstance(member, fpp.ast.DefEnum): + return [constant.name for constant in member.constants] + raise AssertionError(f"fpp synthesized no State enumeration in state machine {self.fpp_name}") + + @property + def signal_constants(self) -> List[str]: + """ The constants of the signal enumeration F Prime generates + + F Prime puts a signal of its own first, for the initial transition, and the modeled signals after it in + name order, which is the order of their values. An action on the initial transition receives F Prime's + signal, so it is bound along with the modeled signals. + """ + return [INITIAL_TRANSITION_SIGNAL] + sorted(signal.name for signal in self._signals) + + @property + def _signals(self) -> List[fpp.ast.DefSignal]: + """ The state machine's signal definitions, in declaration order """ + return [ + signal.definition + for signal in self.state_machine.signals + if isinstance(signal.definition, fpp.ast.DefSignal) + ] + + @property + def signals(self) -> List[Tuple[str, List[ArgumentView]]]: + """ The state machine's signals, as (name, value arguments) pairs in declaration order """ + return [(signal.name, state_machine_values(signal.type_name)) for signal in self._signals] + + @property + def handlers(self) -> List[HandlerView]: + """ The handlers F Prime declares for the state machine's actions and then its guards """ + handlers: List[HandlerView] = [ + StateMachineActionView(self, action.definition) + for action in self.state_machine.actions + if isinstance(action.definition, fpp.ast.DefAction) + ] + handlers += [ + StateMachineGuardView(self, guard.definition) + for guard in self.state_machine.guards + if isinstance(guard.definition, fpp.ast.DefGuard) + ] + return handlers + + +class StateMachineHandlerView(HandlerView): + """ The handler F Prime declares for one action or guard of an internal state machine + + The handler belongs to the state machine rather than to an instance of it, so every instance of the + machine in a component calls the same handler, and the state machine id says which one did. The signal + is the one being handled, the machine's own initial-transition signal on the initial transition. + """ + + #: The word F Prime puts between the machine's name and the member's in the handler name + MEMBER_KIND: str = "" + + def __init__(self, machine: StateMachineView, member: Union[fpp.ast.DefAction, fpp.ast.DefGuard]) -> None: + """ Wrap an action or guard of a state machine """ + self.machine = machine + self.member = member + self.HANDLER_PREFIX = ( + ("SmId", "smId", "The state machine id"), + (f"{machine.nested_class_name}::Signal", "signal", "The signal"), + ) + + @property + def name(self) -> str: + """ The member's name as written in the model """ + return self.member.name + + @property + def description(self) -> str: + """ What the handler is, as it reads in a generated comment """ + return f"Handler for {self.HANDLER_KIND} {self.name} of {self.machine.fpp_name}" + + @property + def handler_name(self) -> str: + """ The name of the handler F Prime declares for this member """ + return f"{self.machine.nested_class_name}_{self.MEMBER_KIND}_{self.name}" + + @property + def parameters(self) -> Sequence[ArgumentView]: + """ The value the member takes, if it declares one """ + return state_machine_values(self.member.type_name) + + +class StateMachineActionView(StateMachineHandlerView): + """ The handler F Prime declares for one action of an internal state machine """ + + HANDLER_KIND = "state machine action" + MEMBER_KIND = "action" + + +class StateMachineGuardView(StateMachineHandlerView): + """ The handler F Prime declares for one guard of an internal state machine + + A guard is an action that decides whether a transition is taken: F Prime declares it `const`, returning + a `bool`, with the same arguments an action has. + """ + + HANDLER_KIND = "state machine guard" + MEMBER_KIND = "guard" + IS_CONST = True + RETURNS_STRICT_BOOL = True + + @property + def handler_return_type(self) -> str: + """ A guard answers whether the transition is taken """ + return "bool" + + +class StateMachineInstanceView: + """ One state machine instance of a component + + F Prime gives each instance a getter for its current state and a method per signal that queues the + signal for the component's thread to handle. + """ + + def __init__(self, analysis: fpp.Analysis, instance: fpp.StateMachineInstance) -> None: + """ Wrap a state machine instance """ + self.instance = instance + self.machine = StateMachineView(analysis, instance.symbol) + + @property + def name(self) -> str: + """ The instance's name, which is also its state machine id """ + return self.instance.name + + @property + def get_state_name(self) -> str: + """ The name of the method F Prime declares to read the instance's current state """ + return f"{self.name}_getState" + + @property + def blocks_when_full(self) -> bool: + """ Whether sending the instance a signal waits for room on a full queue """ + return self.instance.queue_full == fpp.ast.QueueFull.Block + + @property + def send_signals(self) -> List[Tuple[str, List[ArgumentView]]]: + """ The methods F Prime declares to send the instance each signal, as (name, arguments) pairs """ + return [(f"{self.name}_sendSignal_{signal}", values) for signal, values in self.machine.signals] + + class EventView: """ One event of a component """ @@ -450,8 +787,9 @@ class ComponentView: matching that keeps the generated code readable side by side with the component's base class. """ - def __init__(self, component: fpp.Component) -> None: - """ Wrap a component """ + def __init__(self, analysis: fpp.Analysis, component: fpp.Component) -> None: + """ Wrap a component of an analyzed model """ + self.analysis = analysis self.component = component @property @@ -528,27 +866,84 @@ def overflow_hooks(self) -> List[OverflowHookView]: ] return hooks + @cached_property + def state_machine_instances(self) -> List[StateMachineInstanceView]: + """ The component's state machine instances, in name order """ + return [ + StateMachineInstanceView(self.analysis, instance) + for instance in self.component.state_machine_instance_map.values() + ] + + @property + def state_machines(self) -> List[StateMachineView]: + """ The distinct state machines the component instantiates, in the order of their first instance """ + machines: List[StateMachineView] = [] + for instance in self.state_machine_instances: + if all(machine.symbol != instance.machine.symbol for machine in machines): + machines.append(instance.machine) + return machines + + @property + def nested_state_machines(self) -> List[StateMachineView]: + """ The internal state machines defined inside the component, in definition order + + These get no binding from the module they are defined in, which does not descend into components, so the + component's own generation binds them. + """ + symbols = [ + self.analysis.symbol_map[member.node_id] + for member in self.node.members + if isinstance(member, fpp.ast.DefStateMachine) + ] + machines = [StateMachineView(self.analysis, symbol) for symbol in symbols] + return [machine for machine in machines if machine.is_internal] + def check_supported(self) -> None: """ Reject a component whose features this autocoder cannot bind - F Prime declares a pure virtual handler for every state machine action and guard, and for every data - product container. Neither has a Python counterpart yet -- an action takes a state machine id and a - signal, and a container handler takes a `Fw::DpContainer` -- so generating the component anyway would - leave an abstract class and fail at link time, a long way from the cause. + F Prime declares a pure virtual handler for every data product container, and for every action and guard + of an external state machine, and the overflow hook of a state machine instance with `hook` queue-full + behavior. None of these has a Python counterpart -- a container handler takes a `Fw::DpContainer`, an + external machine's actions and guards are declared by its hand-written C++ interface rather than in the + model, and a state machine's overflow hook takes the queued signal as a raw serial buffer -- so + generating the component anyway would leave an abstract class and fail at link time, a long way from + the cause. Raises: - UnsupportedModelError: The component has state machine instances or data products + UnsupportedModelError: The component has data products, or a state machine instance that + cannot be bound """ - unsupported = [ - ("state machine instances", self.component.has_state_machine_instances), - ("data products", self.component.has_data_products), - ] - for feature, present in unsupported: - if present: + remedy = ( + f"Remove the '@ {FPRIME_PYTHON_ANNOTATION}' annotation to implement this component in C++ instead." + ) + if self.component.has_data_products: + raise UnsupportedModelError( + f"Component {self.fpp_name} has data products, which fprime-python cannot bind: F Prime" + f" declares a handler for each of them that has no Python equivalent yet. {remedy}" + ) + for instance in self.state_machine_instances: + machine = instance.machine + described = f"Component {self.fpp_name} has state machine instance {instance.name}" + if not machine.is_internal: + raise UnsupportedModelError( + f"{described} of external state machine {machine.fpp_name}, which fprime-python cannot bind:" + f" its actions and guards are declared by the hand-written C++ interface" + f" {machine.interface_name}, not in the model. Define the state machine's behavior in FPP to" + f" bind it. {remedy}" + ) + if instance.instance.queue_full == fpp.ast.QueueFull.Hook: + raise UnsupportedModelError( + f"{described} with 'hook' queue-full behavior, which fprime-python cannot bind: its overflow" + f" hook {instance.name}_stateMachineOverflowHook takes the signal as a raw FwEnumStoreType and" + f" its value as a raw serial buffer (Fw::SerialBufferBase), which have no Python equivalent." + f" Use 'assert', 'block' or 'drop' instead. {remedy}" + ) + component = machine.enclosing_component + if component is not None and component != self.component.symbol: raise UnsupportedModelError( - f"Component {self.fpp_name} has {feature}, which fprime-python cannot bind: F Prime" - f" declares a handler for each of them that has no Python equivalent yet. Remove the" - f" '@ {FPRIME_PYTHON_ANNOTATION}' annotation to implement this component in C++ instead." + f"{described} of state machine {machine.fpp_name}, which is defined inside another" + f" component, so fprime-python binds its states and signals only with that component. Define" + f" the state machine in a module, or inside {self.fpp_name}, to bind it. {remedy}" ) # Constructing the port views rejects a serial port self._general_ports(inputs=True) @@ -617,15 +1012,22 @@ def handlers(self) -> List[HandlerView]: """ Every handler F Prime declares, members first and overflow hooks last, in generation order Every family gets the same forwarding C++ override and a stub in the Python template (a command's - stub, and its overflow hook's, also sends a response), so the generators walk this one list rather than - each family in turn. + stub, and its overflow hook's, also sends a response, and a guard's returns a bool), so the generators + walk this one list rather than each family in turn. State machine actions and guards come once per + state machine, however many instances of it the component has. """ handlers: List[HandlerView] = list(self.input_ports) handlers += self.internal_ports handlers += self.commands + handlers += self.state_machine_handlers handlers += self.overflow_hooks return handlers + @property + def state_machine_handlers(self) -> List[HandlerView]: + """ The action and guard handlers of every state machine the component instantiates """ + return [handler for machine in self.state_machines for handler in machine.handlers] + @property def events(self) -> List[EventView]: """ The component's events, in id order """ diff --git a/src/fprime_python/autocode/visitor.py b/src/fprime_python/autocode/visitor.py index 05bebcb..8b092a0 100644 --- a/src/fprime_python/autocode/visitor.py +++ b/src/fprime_python/autocode/visitor.py @@ -20,9 +20,10 @@ from .types_generator import ( ArrayBindingGenerator, EnumBindingGenerator, + StateMachineBindingGenerator, StructBindingGenerator, ) -from .view import ComponentView, TopologyView, is_annotated +from .view import ComponentView, StateMachineView, TopologyView, UnsupportedModelError, is_annotated class AnnotatedDefinitionVisitor(fpp.AstVisitor): @@ -34,7 +35,8 @@ class AnnotatedDefinitionVisitor(fpp.AstVisitor): project to implementing it in Python. Definitions nested inside a component are not visited: they belong to the component's own namespace in - C++ and are not reachable as module-level Python types. + C++ and are not reachable as module-level Python types. The exception is an internal state machine, + which F Prime generates into the module's namespace; it is bound with its annotated component. """ def __init__( @@ -72,7 +74,10 @@ def emit(self, files: Dict[str, str]) -> None: files: A mapping of file name to file contents """ for name, contents in files.items(): - self.output[self.output_path / name] = contents + path = self.output_path / name + if path in self.output: + raise UnsupportedModelError(f"Two definitions would both generate {path}; rename one of them") + self.output[path] = contents def visit_DefArray(self, node: fpp.ast.DefArray) -> None: """ Run array generation when an array node is visited """ @@ -99,13 +104,19 @@ def visit_DefStruct(self, node: fpp.ast.DefStruct) -> None: ) def visit_DefStateMachine(self, node: fpp.ast.DefStateMachine) -> None: - """ Skip a state machine definition without descending into it + """ Bind an internal state machine's states and signals, without descending into it + + Like a type, the machine is bound by the module defining it, so that it is bound once however many + components instantiate it. An external machine has no states or signals in the model to bind. fpp's analysis synthesizes a `State` enumeration inside every state machine. F Prime generates that enumeration under a flattened name of its own -- `_StateEnumAc.hpp` rather than - `StateEnumAc.hpp` -- and it is not a module-level type a Python implementation names, so descending - would bind a type that does not exist under a file name that collides between state machines. + `StateEnumAc.hpp` -- so descending would bind a type that does not exist under a file name that + collides between state machines; the state machine binding binds it under F Prime's name instead. """ + machine = StateMachineView(self.analysis, self.symbol_of(node)) + if machine.is_internal: + self.emit(StateMachineBindingGenerator(self.include_manager, machine).files()) def visit_DefModuleTemplate(self, node: fpp.ast.DefModuleTemplate) -> None: """ Skip a module template definition without descending into it @@ -125,12 +136,16 @@ def visit_DefComponent(self, node: fpp.ast.DefComponent) -> None: return symbol = self.symbol_of(node) component = self.analysis.component_map[symbol] + view = ComponentView(self.analysis, component) # Refuse up front rather than generating a class that will not link - ComponentView(component).check_supported() - self.emit(ComponentBindingGenerator(self.include_manager, symbol, component).files()) + view.check_supported() + self.emit(ComponentBindingGenerator(self.include_manager, self.analysis, symbol, component).files()) self.emit( - ComponentImplementationGenerator(self.include_manager, symbol, component).files() + ComponentImplementationGenerator(self.include_manager, self.analysis, symbol, component).files() ) + # A state machine defined inside the component is not reached by visiting the module + for machine in view.nested_state_machines: + self.emit(StateMachineBindingGenerator(self.include_manager, machine).files()) def visit_DefTopology(self, node: fpp.ast.DefTopology) -> None: """ Generate topology bindings when an annotated topology node is visited diff --git a/tests/fixtures/SmExternal.fpp b/tests/fixtures/SmExternal.fpp new file mode 100644 index 0000000..58a050f --- /dev/null +++ b/tests/fixtures/SmExternal.fpp @@ -0,0 +1,14 @@ +module Fixture { + + port Tick(context: U32) + + @ An external state machine: its behavior is hand-written C++ + state machine Outside + + @ fprime-python + queued component PyExternal { + sync input port tick: Tick + state machine instance ext: Outside + } + +} diff --git a/tests/fixtures/SmForeign.fpp b/tests/fixtures/SmForeign.fpp new file mode 100644 index 0000000..0ec771f --- /dev/null +++ b/tests/fixtures/SmForeign.fpp @@ -0,0 +1,25 @@ +module Fixture { + + port Tick(context: U32) + + @ A C++ component defining a state machine of its own + queued component Owner { + sync input port tick: Tick + + state machine Inner { + signal go + initial enter A + state A { on go enter A } + } + + state machine instance own: Inner + } + + @ A Python component instantiating the state machine another component defines + @ fprime-python + queued component PyForeign { + sync input port tick: Tick + state machine instance borrowed: Owner.Inner + } + +} diff --git a/tests/fixtures/SmHook.fpp b/tests/fixtures/SmHook.fpp new file mode 100644 index 0000000..2052df8 --- /dev/null +++ b/tests/fixtures/SmHook.fpp @@ -0,0 +1,22 @@ +module Fixture { + + port Tick(context: U32) + + state machine Toggle { + signal flip + initial enter OFF + state OFF { + on flip enter ON + } + state ON { + on flip enter OFF + } + } + + @ fprime-python + queued component PySmHook { + sync input port tick: Tick + state machine instance toggle: Toggle hook + } + +} diff --git a/tests/fixtures/SmProducts.fpp b/tests/fixtures/SmProducts.fpp new file mode 100644 index 0000000..ef655b2 --- /dev/null +++ b/tests/fixtures/SmProducts.fpp @@ -0,0 +1,23 @@ +module Fw { + port DpGet() + port DpSend() +} + +module Fixture { + + port Tick(context: U32) + + @ A Python component with data products + @ fprime-python + active component PyProducts { + async input port tick: Tick + + product get port productGetOut + product send port productSendOut + time get port timeGetOut + + product record reading: U32 id 0 + product container readings id 0 + } + +} diff --git a/tests/fixtures/SmShared.fpp b/tests/fixtures/SmShared.fpp new file mode 100644 index 0000000..7b3248d --- /dev/null +++ b/tests/fixtures/SmShared.fpp @@ -0,0 +1,10 @@ +module Fixture { + + @ A component instantiating a state machine another module defines + @ fprime-python + active component PyElsewhere { + async input port tick: Tick + state machine instance remote: Blinker + } + +} diff --git a/tests/fixtures/StateMachines.fpp b/tests/fixtures/StateMachines.fpp new file mode 100644 index 0000000..d17975e --- /dev/null +++ b/tests/fixtures/StateMachines.fpp @@ -0,0 +1,104 @@ +module Fixture { + + @ Port driving the state machines + port Tick(context: U32) + + @ An internal state machine shared by every component below + state machine Blinker { + action store: U32 + action reset + guard allow: U32 + signal advance: U32 + signal back + signal rename: string size 8 + initial do { reset } enter IDLE + state IDLE { + on advance if allow do { store } enter BUSY + on rename enter IDLE + } + state BUSY { + on back do { reset } enter IDLE + } + } + + @ An external state machine, which no Python component may instantiate + state machine Outside + + @ A component instantiating one state machine twice and defining another + @ fprime-python + queued component PySm { + sync input port tick: Tick + + @ A state machine defined inside the component + state machine Local { + action note + signal go + initial enter X + state X { + on go do { note } enter Y + } + state Y + } + + state machine instance first: Blinker priority 1 assert + state machine instance second: Blinker priority 2 drop + state machine instance loc: Local + } + + @ A second component sharing the Blinker state machine + @ fprime-python + active component PyShared { + async input port tick: Tick + state machine instance only: Blinker + } + + @ An alias of a primitive type, which a state machine passes by const reference + type Count = U32 + + @ An enumeration carried by a signal + enum Mode { ON, OFF } + + @ A structure carried by a signal + struct Pair { left: U32, right: U32 } + + @ A state machine whose signals carry every kind of value, with a nested state and a choice + state machine Wide { + action onCount: Count + action onMode: Mode + action onPair: Pair + action onLabel: string + guard checkCount: Count + guard checkMode: Mode + signal count: Count + signal mode: Mode + signal pair: Pair + signal label: string + signal hop + initial enter OUTER + state OUTER { + initial enter INNER + state INNER { + on count do { onCount } enter DECIDE + on label do { onLabel } + } + on pair do { onPair } enter OUTER + } + choice DECIDE { if checkCount enter OUTER else enter LAST } + state LAST { + on mode if checkMode do { onMode } enter OUTER + on hop enter LAST + } + } + + @ A component instantiating the wide state machine with 'block' queue-full behavior + @ fprime-python + queued component PyWide { + sync input port tick: Tick + + @ An external state machine defined but never instantiated, so nothing binds it + state machine Hidden + + state machine instance wide: Wide priority 1 block + } + +} diff --git a/tests/test_generator.py b/tests/test_generator.py index 67cae6f..027a43c 100644 --- a/tests/test_generator.py +++ b/tests/test_generator.py @@ -1,6 +1,7 @@ """ Tests of the bindings generated for a small fixture model """ from __future__ import annotations +import json from pathlib import Path from typing import Dict @@ -9,6 +10,7 @@ from fprime_python.autocode.__main__ import main from fprime_python.autocode.include import IncludeManager from fprime_python.autocode.model import load_model +from fprime_python.autocode.view import UnsupportedModelError from fprime_python.autocode.visitor import AnnotatedDefinitionVisitor FIXTURES = Path(__file__).parent / "fixtures" @@ -173,3 +175,256 @@ def test_members_without_hook_get_no_overflow_hook(files: Dict[str, str], hooks: assert "PLAIN_cmdOverflowHook" not in hooks["PyHooks.template.py"] assert "overflowHook" not in files["PyQueued.cpp"] assert "overflowHook" not in files["PyQueued.template.py"] + + +@pytest.fixture +def machines(tmp_path: Path) -> Dict[str, str]: + return generate("StateMachines.fpp", tmp_path) + + +def handler_body(source: str, call: str) -> str: + """ The text of the generated handler definition leading up to a call it makes """ + return source[source.rindex(" ::", 0, source.index(call)):source.index(call)] + + +def test_state_machine_actions_and_guards_are_declared(machines: Dict[str, str]) -> None: + header = machines["PySm.hpp"] + assert ( + "void Fixture_Blinker_action_store(\n" + " SmId smId, //!< The state machine id\n" + " Fixture_Blinker::Signal signal, //!< The signal\n" + " U32 value\n" + " ) override;" + ) in header + assert "void Fixture_Blinker_action_reset(" in header + # A guard is const and answers with a bool, as F Prime declares it + assert ( + "bool Fixture_Blinker_guard_allow(\n" + " SmId smId, //!< The state machine id\n" + " Fixture_Blinker::Signal signal, //!< The signal\n" + " U32 value\n" + " ) const override;" + ) in header + # A machine defined inside the component is named as F Prime names it, with the component folded in + assert "void Fixture_PySm_Local_action_note(" in header + # Two instances of one machine share its handlers + assert header.count("Fixture_Blinker_action_store(") == 1 + + +def test_state_machine_handlers_are_forwarded_into_python(machines: Dict[str, str]) -> None: + source = machines["PySm.cpp"] + calls = [ + 'm_self.attr("Fixture_Blinker_action_store")(smId, signal, value);', + 'm_self.attr("Fixture_Blinker_action_reset")(smId, signal);', + 'm_self.attr("Fixture_PySm_Local_action_note")(smId, signal);', + 'pybind11::object return_value = m_self.attr("Fixture_Blinker_guard_allow")(smId, signal, value);', + ] + for call in calls: + assert "pybind11::gil_scoped_acquire acquired{};" in handler_body(source, call) + assert "bool PySm ::\n Fixture_Blinker_guard_allow(" in source + guard = source[source.index('attr("Fixture_Blinker_guard_allow")'):] + guard = guard[:guard.index("\n }")] + # Anything but a bool is refused rather than converted, None included + assert "if (!pybind11::isinstance(return_value)) {" in guard + assert 'throw pybind11::type_error("PySm.Fixture_Blinker_guard_allow must return a bool");' in guard + assert "return return_value.cast();" in guard + + +def test_missing_state_machine_handlers_are_reported_at_init(machines: Dict[str, str]) -> None: + source = machines["PySm.cpp"] + init = source[source.index('.attr("_init_ac")(this);'):source.index("PySmComponentBase::init(")] + expected = [ + ("action", "Fixture_Blinker_action_store"), + ("action", "Fixture_Blinker_action_reset"), + ("guard", "Fixture_Blinker_guard_allow"), + ("action", "Fixture_PySm_Local_action_note"), + ] + for kind, handler in expected: + check = init[init.index(f'if (!pybind11::hasattr(pybind11::type::of(this->m_self), "{handler}")) {{'):] + check = check[:check.index("}")] + assert "this->m_self = pybind11::none();" in check + assert f'throw pybind11::attribute_error("PySm does not implement state machine {kind} {handler}");' in check + + +def test_state_machine_handlers_get_python_stubs(machines: Dict[str, str]) -> None: + template = machines["PySm.template.py"] + assert "def Fixture_Blinker_action_store(self, smId, signal, value):" in template + assert "def Fixture_Blinker_action_reset(self, smId, signal):" in template + assert "def Fixture_PySm_Local_action_note(self, smId, signal):" in template + start = template.index("def Fixture_Blinker_guard_allow(self, smId, signal, value):") + guard = template[start:template.index("\n def ", start)] + assert "returning a bool. Returning False (this stub) blocks the transition" in guard + assert "return False" in guard + + +def test_state_machine_instance_methods_are_bound(machines: Dict[str, str]) -> None: + header = machines["PySm.hpp"] + binding = machines["PySmBindingAc.cpp"] + for instance in ("first", "second"): + for method in ("getState", "sendSignal_advance", "sendSignal_back", "sendSignal_rename"): + assert f"using PySmComponentBase::{instance}_{method};" in header + assert f'.def("{instance}_getState", &Fixture::PySm::{instance}_getState)' in binding + assert f'.def("{instance}_sendSignal_advance", &Fixture::PySm::{instance}_sendSignal_advance)' in binding + # A sized string signal is checked against its declared size before F Prime serializes it + assert ( + f'.def("{instance}_sendSignal_rename", [](Fixture::PySm& self, const Fw::StringBase& value) {{\n' + " FprimePython::check_declared_size(value, 8);\n" + f" return self.{instance}_sendSignal_rename(value);\n" + " })" + ) in binding + assert '.def("loc_sendSignal_go", &Fixture::PySm::loc_sendSignal_go)' in binding + + +def test_state_machine_ids_are_bound_in_the_component(files: Dict[str, str], machines: Dict[str, str]) -> None: + assert "using PySmComponentBase::SmId;" in machines["PySm.hpp"] + binding = machines["PySmBindingAc.cpp"] + assert 'pybind11::class_ component(m, "PySm");' in binding + assert ( + 'pybind11::native_enum(component, "SmId", "enum.Enum")\n' + ' .value("first", Fixture::PySm::SmId::first)\n' + ' .value("loc", Fixture::PySm::SmId::loc)\n' + ' .value("second", Fixture::PySm::SmId::second)\n' + " .finalize();" + ) in binding + assert "#include " in binding + # Components without state machines are bound as before + assert "SmId" not in files["PyQueuedBindingAc.cpp"] + assert 'pybind11::class_(m, "PyQueued")' in files["PyQueuedBindingAc.cpp"] + + +def test_state_machine_states_and_signals_are_bound_once_per_machine(machines: Dict[str, str]) -> None: + binding = machines["BlinkerStateMachineBindingAc.cpp"] + assert '#include "BlinkerStateMachineAc.hpp"' in binding + assert "void init_Fixture_Blinker_StateMachine(pybind11::module_& m) {" in binding + assert 'pybind11::class_ enumeration(m, "Blinker_State");' in binding + assert ( + 'pybind11::native_enum(enumeration, "T", "enum.Enum")\n' + ' .value("__FPRIME_UNINITIALIZED", Fixture::Blinker_State::__FPRIME_UNINITIALIZED)\n' + ' .value("BUSY", Fixture::Blinker_State::BUSY)\n' + ' .value("IDLE", Fixture::Blinker_State::IDLE)\n' + " .export_values()\n" + " .finalize();" + ) in binding + # The initial-transition signal is bound too: an action on the initial transition receives it + signal = "Fixture::BlinkerStateMachineBase::Signal" + assert ( + f'pybind11::native_enum<{signal}>(m, "Blinker_Signal", "enum.Enum")\n' + f' .value("__FPRIME_INITIAL_TRANSITION", {signal}::__FPRIME_INITIAL_TRANSITION)\n' + f' .value("advance", {signal}::advance)\n' + f' .value("back", {signal}::back)\n' + f' .value("rename", {signal}::rename)\n' + " .finalize();" + ) in binding + assert json.loads(machines["BlinkerStateMachineBinding.json"]) == { + "Fixture": "(void) init_Fixture_Blinker_StateMachine(fprime_Fixture);" + } + # Two instances and two components share Blinker, and it is still registered exactly once + registrations = [ + name for name, contents in machines.items() if '"Blinker_State"' in contents or '"Blinker_Signal"' in contents + ] + assert registrations == ["BlinkerStateMachineBindingAc.cpp"] + # An external machine has nothing to bind + assert not [name for name in machines if name.startswith("Outside")] + + +def test_nested_state_machine_is_bound_under_its_module(machines: Dict[str, str]) -> None: + binding = machines["PySm_LocalStateMachineBindingAc.cpp"] + assert '#include "PySm_LocalStateMachineAc.hpp"' in binding + assert 'pybind11::class_ enumeration(m, "PySm_Local_State");' in binding + assert 'pybind11::native_enum(m, "PySm_Local_Signal"' in binding + assert json.loads(machines["PySm_LocalStateMachineBinding.json"]) == { + "Fixture": "(void) init_Fixture_PySm_Local_StateMachine(fprime_Fixture);" + } + + +def test_state_machine_defined_elsewhere_is_not_bound_again(tmp_path: Path) -> None: + model = load_model( + [FIXTURES / "SmShared.fpp"], [FIXTURES / "Config.fpp", FIXTURES / "StateMachines.fpp"] + ) + visitor = AnnotatedDefinitionVisitor(tmp_path, model.analysis, IncludeManager([FIXTURES])) + files = {path.name: contents for path, contents in visitor.generate(model).items()} + assert "void Fixture_Blinker_action_store(" in files["PyElsewhere.hpp"] + assert not [name for name in files if "StateMachineBinding" in name] + + +def test_external_state_machine_instance_is_rejected(tmp_path: Path) -> None: + with pytest.raises(UnsupportedModelError) as error: + generate("SmExternal.fpp", tmp_path) + message = str(error.value) + assert "Component Fixture.PyExternal has state machine instance ext of external state machine" in message + assert "hand-written C++ interface Fixture::Outside_Interface, not in the model" in message + + +def test_state_machine_instance_with_hook_is_rejected(tmp_path: Path) -> None: + with pytest.raises(UnsupportedModelError) as error: + generate("SmHook.fpp", tmp_path) + message = str(error.value) + assert "Component Fixture.PySmHook has state machine instance toggle with 'hook' queue-full behavior" in message + assert ( + "toggle_stateMachineOverflowHook takes the signal as a raw FwEnumStoreType and its value as a raw serial buffer" + ) in message + + +def test_state_machine_values_follow_f_prime_parameter_spelling(machines: Dict[str, str]) -> None: + header = machines["PyWide.hpp"] + # F Prime passes a value by value only when its declared type is primitive: an alias of U32 goes by reference + assert ( + "void Fixture_Wide_action_onCount(\n" + " SmId smId, //!< The state machine id\n" + " Fixture_Wide::Signal signal, //!< The signal\n" + " const Fixture::Count& value\n" + " ) override;" + ) in header + assert " const Fixture::Mode& value\n ) override;" in header + assert " const Fixture::Pair& value\n ) override;" in header + assert " const Fw::StringBase& value\n ) override;" in header + assert " const Fixture::Count& value\n ) const override;" in header + assert " const Fixture::Mode& value\n ) const override;" in header + assert "Fixture::Count value" not in header + + +def test_state_machine_with_nested_states_and_choice_is_bound(machines: Dict[str, str]) -> None: + binding = machines["WideStateMachineBindingAc.cpp"] + for state in ("__FPRIME_UNINITIALIZED", "LAST", "OUTER_INNER"): + assert f'.value("{state}", Fixture::Wide_State::{state})' in binding + # Signals are bound in the order of their values, which F Prime assigns by name + signal = "Fixture::WideStateMachineBase::Signal" + names = ["__FPRIME_INITIAL_TRANSITION", "count", "hop", "label", "mode", "pair"] + assert "".join(f' .value("{name}", {signal}::{name})\n' for name in names) in binding + assert "void Fixture_Wide_guard_checkCount(" not in machines["PyWide.hpp"] + assert "bool Fixture_Wide_guard_checkCount(" in machines["PyWide.hpp"] + + +def test_blocking_state_machine_signals_release_the_gil(machines: Dict[str, str]) -> None: + binding = machines["PyWideBindingAc.cpp"] + release = "pybind11::call_guard()" + assert f'.def("wide_sendSignal_hop", &Fixture::PyWide::wide_sendSignal_hop, {release})' in binding + assert f'.def("wide_sendSignal_count", &Fixture::PyWide::wide_sendSignal_count, {release})' in binding + # A string is checked while the GIL is still held, then the GIL is released for the send + assert ( + '.def("wide_sendSignal_label", [](Fixture::PyWide& self, const Fw::StringBase& value) {\n' + " FprimePython::check_declared_size(value, FW_FIXED_LENGTH_STRING_SIZE);\n" + " pybind11::gil_scoped_release released{};\n" + " return self.wide_sendSignal_label(value);\n" + ) in binding + # Instances that do not block keep the GIL + assert "gil_scoped_release" not in machines["PySmBindingAc.cpp"] + + +def test_unbound_external_state_machines_get_no_bindings(machines: Dict[str, str]) -> None: + assert not [name for name in machines if name.startswith(("Outside", "PyWide_Hidden", "Hidden"))] + + +def test_state_machine_defined_in_another_component_is_rejected(tmp_path: Path) -> None: + with pytest.raises(UnsupportedModelError) as error: + generate("SmForeign.fpp", tmp_path) + message = str(error.value) + assert ( + "Component Fixture.PyForeign has state machine instance borrowed of state machine Fixture.Owner.Inner" + ) in message + assert "which is defined inside another component" in message + + +def test_data_products_are_rejected(tmp_path: Path) -> None: + with pytest.raises(UnsupportedModelError, match="Component Fixture.PyProducts has data products"): + generate("SmProducts.fpp", tmp_path)