Modernize cabbage: spec-conformant Cucumber runner on gherkin 3.0 (1.0.0 proposal) - #98
Open
tomasz-tomczyk wants to merge 64 commits into
Open
Modernize cabbage: spec-conformant Cucumber runner on gherkin 3.0 (1.0.0 proposal)#98tomasz-tomczyk wants to merge 64 commits into
tomasz-tomczyk wants to merge 64 commits into
Conversation
Depend on the gherkin fork (path dep for local iteration; final form is the git
dep on the public-api branch) and consume Gherkin.pickles/2 in the Loader. The
pickle is already fully resolved, so cabbage's own flattening is gone:
- Removed Gherkin.parse |> Gherkin.flatten |> fix_step_types. Outline expansion,
feature- AND rule-level background prepending, tag inheritance/unioning,
conjunction (And/But/*) keyword resolution and <placeholder> substitution all
come from the pickle now.
- The Loader is a thin projection from pickles onto cabbage-owned structs
(Cabbage.Feature.{Document, Scenario, Step}), replacing the gherkin
Gherkin.Elements.* structs the fork no longer ships. Step type
Context/Action/Outcome -> display keyword Given/When/Then; data tables ->
header-keyed maps; doc strings -> content + trailing newline.
This unlocks two long-standing issues for free, now covered by loader_test:
- Background steps run before each scenario's steps (cabbage-ex#68).
- Scenarios under a Rule run, with the rule-level background applied,
after the feature background (cabbage-ex#69).
Behaviour changes (all from adopting conformant Gherkin):
- Outline pickles share the scenario name per spec; the Loader numbers
duplicates as "<name> (Example N)" to keep ExUnit names unique (matches the
prior naming, so feature_attributes_test is unchanged).
- The non-standard valued-tag syntax (@tag_with_value my_value / @timeout 100)
is invalid Gherkin and is now rejected. tags.feature uses a plain @valued_tag;
timeout.feature uses @slow and sets the per-scenario timeout via the standard
@moduletag timeout: 100. Hardened Helpers.run_tag/4 to ignore non-name
(valued) ExUnit tags rather than crashing on them.
All cabbage tests green (55).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switches from the local path dep used during development to a git dep on tomasz-tomczyk/gherkin master, now that the public API is pushed.
Convert the language-neutral cucumber/cucumber-expressions YAML test suite (commit 0555a711) to JSON for matching, parser, tokenizer, transformation and regular-expression categories (115 files). Loaded at test time via the built-in Elixir 1.18 JSON module. Refs cabbage-ex#47
Add the official cross-language testdata (parsing, evaluations, errors) converted from upstream YAML to JSON so it can be loaded with the built-in JSON module (no jason dependency). UPSTREAM.md records the source commit SHA 28a5e5e97900b8e6e13d4517a2f24337c6c686fd and the Ruby conversion command. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement the Cucumber tag expression boolean language (and/or/not, grouping, backslash escaping, empty-is-true) as a self-contained engine in an isolated namespace so it can later be extracted to its own Hex package. - Tokenizer + shunting-yard parser ported faithfully from the reference implementation, including verbatim syntax-error messages. - AST structs with a Node protocol for evaluation and String.Chars for the canonical fully-parenthesised to_string/1 form. - Public API: parse/1, parse!/1, evaluate/2; SyntaxError exception. - Unit tests covering parsing, rendering, round-tripping, evaluation, errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prove Cabbage.TagExpression against the vendored cross-language corpus. - Conformance harness runs all three corpora and returns pass/fail tallies. - `mix conformance.tags` prints the scoreboard (registered with preferred_envs so it runs in :test); `mix conformance` runs the per-corpus ExUnit assertions. - Conformance test tagged :conformance and excluded from the default `mix test` so the existing suite stays green. Result: parsing 23/23, evaluations 26/26, errors 15/15 (100%). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the reference cucumber/cucumber-expressions design into an isolated,
self-contained namespace (Cabbage.CucumberExpression.*) so it can later be
extracted to its own Hex package:
- Tokenizer: source -> token maps (START/END_OF_LINE, WHITE_SPACE, BEGIN/END
parameter+optional, ALTERNATION, TEXT) with codepoint offsets and escape
handling.
- Parser: layered recursive descent -> AST node maps (EXPRESSION/ALTERNATION/
ALTERNATIVE/OPTIONAL/PARAMETER/TEXT) matching the testdata shape exactly.
- ParameterType + ParameterTypeRegistry: built-in int/float/word/string/{}
plus double/byte/short/long/biginteger/bigdecimal with reference regexps and
transforms.
- TreeRegexp/Group: reconstructs the capture-group tree so each parameter's
value (and sub-groups) feed its transform; non-participating groups -> nil.
- CucumberExpression: AST -> anchored regex (with structural validation) and
text -> transformed args. Error messages mirror the reference wording.
Includes engine unit tests.
Refs cabbage-ex#47
- test/support grader loads each vendored JSON case and grades it per category (matching/parser/tokenizer/transformation/regex), handling both success and expected-exception cases. - mix conformance.expressions prints the per-category scoreboard; declared in def cli preferred_envs so it runs in :test without a MIX_ENV warning. - mix conformance alias runs the tagged :conformance ExUnit suite, which is excluded from the default mix test run (kept green) via test_helper. Engine is fully conformant: matching 62/62, parser 27/27, tokenizer 15/15, transformation 8/8, regex 3/3. Refs cabbage-ex#47
String step patterns now resolve in priority order:
1. legacy named-capture {name:type} -> Cabbage.Feature.CucumberExpression
(unchanged; still produces named vars)
2. a standard Cucumber Expression containing a {...} parameter -> the new
Cabbage.CucumberExpression engine (anonymous params, optional text (s),
alternation a/b)
3. otherwise an exact literal match (issue cabbage-ex#64 preserved)
Opting in to (2) requires a {...} parameter so patterns like
'It costs $5 (USD)' stay literal. Adds an end-to-end feature test exercising
{int}, (s) and cuke/banana alternation.
Refs cabbage-ex#47
# Conflicts: # mix.exs # test/test_helper.exs
Vendor the targeted CCK sample areas (.feature/.feature.md inputs, golden .ndjson streams, media) from cucumber/compatibility-kit @ 8fa40701, plus the reference TypeScript step definitions (kept for reference, not shipped). Pin recorded in UPSTREAM.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A direct interpreter over Gherkin pickles, separate from Cabbage.Feature's compile-time ExUnit generation. Parses a feature to pickles, runs each pickle's steps against a StepRegistry, and emits the full cucumber-messages Envelope stream (StepDefinition, TestRun*, TestCase*, TestStep*, suggestion) reusing the gherkin dep's Source/GherkinDocument/Pickle envelopes and key-sorted NDJSON serializer. - StepRegistry: ordered cucumber-expression / regex step definitions. - Matcher: per-step match producing stepMatchArguments (recursive group tree with start/value + parameterTypeName) and transformed values; 0/1/>1 matches map to undefined/defined/ambiguous. - Normalizer: faithful port of cucumber-js comparison (drop meta, recursively drop ignorable keys, reorder testRunHook envelopes, sort unordered groups). - TreeRegexp.match_with_index/2: additive helper exposing group start offsets. Status rules mirror fake-cucumber: "pending"/"skipped" returns, raises map to FAILED (AssertionError vs Error), first non-passed step skips the rest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Score the message-emitting runner against the vendored CCK goldens: - conformance.cck Mix task prints CCK x/44 plus per-sample PASS/FAIL and the deferred (unsupported) area list; registered in cli preferred_envs :test. - Runner drives Cabbage.Messages over each sample, normalizes both sides, and diffs them; resilient to parser errors so the scoreboard never crashes. - Steps re-implements the reference step definitions in Elixir per area. - cck_conformance_test asserts the passing areas, skipping the two blocked on gherkin-parser gaps (minimal description bullets, markdown dialect). - test/conformance/cck added to :test elixirc_paths. CCK: 15/44 (15 of 17 targeted areas green; minimal + markdown blocked upstream). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Cabbage.PendingError / Cabbage.SkippedError so step defs can signal PENDING/SKIPPED via a raised exception (carrying the reference PendingException / SkippedException type), distinct from the "pending"/"skipped" return values which carry no exception. Rework Cabbage.Messages step execution to compute each step's intrinsic status (undefined / ambiguous / run result) and then apply cucumber-js skip propagation: a prior failed-ish step skips later *executable* steps but leaves undefined/ambiguous intact, while a real intrinsic skip cascades to every later step. This matches the CCK failedish-combinations and all-statuses goldens. Ambiguity detection (cabbage-ex#88) is surfaced here only; the compile-time Cabbage.Feature first-match path is left unchanged to avoid breaking existing feature modules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vendor the all-statuses, failedish-combinations, stack-traces, pending-exception and skipped-exception samples (corpus SHA unchanged) and add their Elixir step definitions, mirroring the reference fake-cucumber TS. pending-exception/skipped-exception raise the new Cabbage.PendingError/SkippedError; all-statuses/failedish use anchored regex step defs so the emitted stepDefinition pattern sources match the goldens byte-for-byte. Also vendor test-run-exception but keep it deferred: its golden asserts a run-level crash rather than a normal run (cucumber-js marks it UNSUPPORTED too). The scoreboard now prints a reason for each documented deferral. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Picks up gherkin master 33637da (descriptions-with-bullets + Markdown dialect). Cabbage.Messages now passes markdown: true to Gherkin when the source format is :markdown. CCK: minimal now passes (20 -> 21). markdown remains gated on the attachments wave (its scenario attaches data).
Lists the {type} parameter names referenced by an expression without
requiring them to be registered, so callers can detect an undefined
parameter type before compile/2 would raise.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nner
StepRegistry.define_parameter_type/2 registers a domain-specific {type}
(name, regexp, transformer) into the backing ParameterTypeRegistry and
records it for the parameterType message. A Cucumber Expression that
references an unregistered {type} no longer crashes registration: it
becomes an :undefined step definition (never matches, reported UNDEFINED)
and the offending type is recorded for the undefinedParameterType message.
Cabbage.Messages now emits parameterType envelopes (one per custom type),
undefinedParameterType envelopes (no id), and skips stepDefinition for
undefined defs. Regular-expression step defs already matched; their group
offsets/values flow into stepMatchArguments with empty groups for
non-participating optional captures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (CCK 21 -> 24)
Vendor the three sample areas (feature + golden ndjson + reference ts) from
the pinned CCK SHA, add their Elixir step registries (including the custom
{flight} parameter type and the regex step def), and target them in the
runner. All three pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the 9 hook CCK areas from cucumber/compatibility-kit at the same pinned commit (8fa40701) as the existing corpus: hooks, hooks-named, hooks-conditional, hooks-skipped, hooks-undefined, global-hooks, global-hooks-beforeall-error, global-hooks-afterall-error, and skipped-failing-hook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces Cabbage.Messages.HookRegistry (before/after-scenario and
BeforeAll/AfterAll hooks, optionally tag-scoped or named) and teaches
Cabbage.Messages to emit the hook envelope stream:
* hook definition envelopes, split into a before-block (emitted ahead
of step defs) and after-block (after them) to match the reference's
source-order registration section;
* scenario before/after hooks woven into each testCase as hookId test
steps, wrapped in testStepStarted/testStepFinished;
* BeforeAll/AfterAll as testRunHookStarted/testRunHookFinished, with
BeforeAll after testRunStarted and AfterAll (reverse order) before
testRunFinished;
* skip/fail propagation mirroring fake-cucumber: a failing Before skips
the steps but After hooks still run; a skipped Before cascades to
later Before hooks and steps; a skipped After only marks itself; a
failed BeforeAll aborts test-case execution; any failed hook makes
testRunFinished.success false.
Tag-scoped hooks evaluate their expression against the pickle tags via
Cabbage.TagExpression.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds hook step definitions and hook registrations (mirroring the reference TypeScript) to the CCK Steps fixture, passes the hook registry through the runner, targets the 9 hook areas, and updates the conformance test's passing list and the scoreboard's deferral notes (only attachment-dependent hook areas remain deferred). CCK: 21/44 -> 30/44. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # lib/cabbage/messages.ex # test/conformance/cck/runner.ex # test/conformance/cck/steps.ex
…defs, binaries) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce Cabbage.Messages.Attach, a per-run Agent collector that step and
hook bodies push attachments onto via the reserved :__attach__ world key.
The runner threads the collector through each step/hook, drains it after the
body runs (even on a raise), and emits an attachment envelope per attachment
between testStepStarted/testStepFinished (scenario) or
testRunHookStarted/testRunHookFinished (global hooks).
Text bodies are IDENTITY; byte bodies ({:bytes, binary}) are BASE64. log/2
and link/2 cover the cucumber log + uri-list media types; fileName is carried
for renamed attachments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add step/hook definitions for attachments, examples-tables-attachment, hooks-attachment and global-hooks-attachments, reading the vendored binary fixtures so base64 bodies match the goldens exactly. Wire the markdown step's this.log attachment. markdown stays blocked on the gherkin MDG description gap (envelope count + both attachments now match; only feature.description diverges). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Since Elixir 1.19 two Regex structs from separate compilations no longer compare equal (the embedded compiled pattern differs), so asserting struct equality against ~r/\d+/ fails. Match the struct and compare Regex.source/1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matrix builds via erlef/setup-beam (strict versions) on Elixir 1.18.1/OTP 27, 1.19.5/OTP 28, and 1.20.0-rc.6/OTP 27 (the set-theoretic type checker). Caches deps and _build, then runs deps.get, format --check-formatted, compile --warnings-as-errors, test, the conformance scoreboards, and the tagged conformance ExUnit suite as the hard regression gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Describe Cabbage as a spec-conformant Cucumber runner on the gherkin fork, headline the conformance scores (CCK 43/44, Cucumber Expressions 115/115, Tag Expressions 64/64), document Elixir 1.18+ / built-in JSON / gherkin as the only runtime dep, swap the dead coveralls/CircleCI badges for the GitHub Actions badge, and add fork attribution (cabbage-ex/cabbage, MIT). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spec Cabbage.base_path/0 and global_tags/0 (and give the module a real @moduledoc), and document/spec Feature.compile_step/3. The messages runner, tag/cucumber expression engines, and the messages registries were already fully specced. Verified clean under the Elixir 1.20 set-theoretic type checker with --warnings-as-errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rewrite the normalizer testRunHook reorder comment to describe the load-bearing canonicalization (hooks have landed) instead of claiming it is a no-op. - Validate Cabbage.Messages.run/3 per-feature opts against [:uri, :format] and raise on unknown keys (notably :order, which is run_features-only) rather than silently dropping them. Add tests pinning the raise. - Remove the Cabbage.Messages reference from TreeRegexp's @moduledoc so the cucumber-expression sub-engine has zero references to the rest of cabbage. - Document suggestion_snippets/1's count heuristic as intentionally corpus-fitted and spell out what would break it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the parser-envelope reduce (parser_envelopes/2), the whole registration-envelope block (registration_envelopes/3, returning the ordered sections plus the hook_ids/by_definition lookup maps), and the plan+execute block (plan_and_execute/9) so the final ++ chain in do_run_features is the only thing left after gathering named sections — making the cucumber-messages ordering contract a readable manifest. Also extract run_attempt/5's step-execution reduces into execute_steps/3. Behaviour-preserving: CCK output is byte-identical (43/44 unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…x#88) The compile-time runner matches scenario steps with silent first-match-wins and there was no test pinning that behaviour. Add: - a characterization test pinning the current behaviour (the last textually-written matching definition wins, since @steps accumulates by prepending) so a future refactor cannot silently change it; - an opt-in, non-breaking check: use Cabbage.Feature, on_ambiguous_step: :ignore | :warn | :raise (DEFAULT :ignore, preserving all shipped behaviour). When :warn/:raise, __before_compile__ detects scenario steps matched by >1 registered pattern and warns / raises CompileError; - validation of the option value (ArgumentError on anything else); - docs for the new option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The three Feature.{CucumberExpression,Parameter,ParameterType} modules were
an old parallel cucumber-expression translator kept alive only for helpers.ex's
{name:type} named-capture sugar. Clean removal is NOT achievable without a
behaviour change: the full Cabbage.CucumberExpression spec engine produces
ANONYMOUS captures and rejects {name:type} outright (':' is reserved in a spec
parameter-type name), so it cannot bind a value to a caller-named variable the
way {count:int} -> (?<count>\d+) does.
So consolidate the trio into a single, clearly-documented
Cabbage.Feature.NamedCaptureExpression and route helpers.ex through it. The
module docs spell out that this is an INTENTIONAL frozen mini-engine for the
cabbage-specific named-capture sugar, not accidental duplication. The two old
unit tests are replaced by one consolidated test (same behavioural assertions
plus a binding round-trip).
All gates unchanged: 205-equivalent tests green, CCK 43/44, tags 64/64,
expressions 115/115.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Graduate to 1.0.0: the gherkin rewrite, conformance maturity, and the
breaking changes (Elixir/gherkin floors, valued-tag rejection, Background
now executing) justify a major. Keep the working git gherkin dep so the
build stays green, with a RELEASE comment to swap to {:gherkin, "~> 3.0"}
once gherkin 3.0.0 is on hex.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CHANGELOG 1.0.0 entry documents breaking changes (Elixir ~>1.18, gherkin ~>3.0, valued-tag rejection, Background now runs, internal struct renames), the unchanged public macro API, and the added pickle/expressions/tags/ messages/conformance features. RELEASING.md gives the strict publish order (gherkin 3.0.0 first, then swap cabbage dep and publish 1.0.0) with gate commands. UPGRADING.md is the 0.4.1 -> 1.0.0 guide emphasising step-def code is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The CCK, cucumber-expressions, and tag-expressions corpora are vendored from cucumber/* (all MIT). Add each upstream LICENSE verbatim as LICENSE.upstream in its corpus dir and note it in the corresponding UPSTREAM.md. The CCK LICENSE also covers the reference/*.ts files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Feature runner compiled each step to read/write a per-scenario Agent
registered under a global name. The Agent was never load-bearing (steps run
inline and sequentially in one test) and caused two problems: the global name
could collide and leak state between same-named scenarios under async: true, and
the Agent was started but never stopped.
Compile each step to a fn context -> next_context end and thread state through
them with a reduce. State is now a per-invocation value, so async: true is safe
by construction and no processes leak. The public step API is unchanged: the
{:ok, delta} merge return, assertion-only steps, vars/state args, tag seeding,
data tables and doc strings all behave as before.
Remove the now-unused Cabbage.Feature.Helpers Agent helpers (start_state,
fetch_state, update_state, agent_name) and make tag-state collection pure.
Introduce an ExUnit formatter that writes a cucumber-messages NDJSON stream while a normal `mix test` run executes Cabbage.Feature scenarios. For each scenario it emits meta, per-feature source/gherkinDocument/pickle, testRunStarted, and per-scenario testCase/testCaseStarted/testStepStarted/ testStepFinished/testCaseFinished, closing with testRunFinished. Parser-side envelopes reuse the gherkin dependency's Gherkin.Message builders. Step results are attributed at scenario granularity (cabbage runs every step in one ExUnit test); granular per-step attribution is a follow-up. Expose __cabbage_document__/0 on feature modules so the formatter can resolve each scenario back to its feature file and pickle. Output path is configurable via `config :cabbage, messages_output` or the formatter's :messages_output option, defaulting to cucumber-messages.ndjson.
Address review feedback comparing next vs upstream/master ahead of the
cabbage-ex/cabbage 1.0.0 proposal:
- Drop the non-standard `{name:type}` named-capture step sugar (not in the
Cucumber Expressions spec). Remove the module + its routing in helpers.ex
and the legacy-only tests; such patterns now flow to the spec engine and
raise a clear compile-time error. Documented as a breaking change in
CHANGELOG and UPGRADING.md (migrate to `~r/(?<name>...)/`).
- ci.yml: bump actions (checkout@v6, cache@v5); Elixir 1.20.1 (now released)
- CHANGELOG: fold the Unreleased section into 1.0.0 (never published)
- README: lead with Example Usage; drop Requirements/Attribution prose
- mix.exs: remove the package-name NOTE comment
- cucumber_expression.ex: drop the issue reference from the moduledoc
- Move ROADMAP.md and RELEASING.md out of the repo (now under ../)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Low-risk, conformance-gated cleanup from the cabbage audit: - Signpost the two runners. `Cabbage` and `Cabbage.Feature` moduledocs now explain Feature (compile-time, what you use) vs Messages (runtime CCK interpreter) vs Formatter, and disambiguate `Cabbage.Messages` from the dep's `Gherkin.Message`. Add a "Code map" to `Cabbage.Messages`. - Dedup the two step-finders in feature.ex via `eval_regex/1` + `step_matches?/2` (also reused in compile/3 and the ambiguity reporter). - Unify the three drop-nil map helpers (maybe_put ×2 + put_optional) into `Cabbage.Messages.MapHelpers.put_unless_nil/3`. - Drop remaining issue-number references from comments/moduledocs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…3.0 proposal branch Repoint source_url/homepage_url and the GitHub package link at the canonical cabbage-ex/cabbage repo ahead of the proposal PR. Flip the gherkin git dep from the fork's stale `master` to `next`, which carries the gherkin 3.0 rewrite (cabbage-ex/gherkin#23). This lets cabbage build/test against the rewritten parser before that PR merges; it becomes {:gherkin, "~> 3.0"} at release (see RELEASING.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
defgiven/defwhen/defthen now accept a string Cucumber Expression pattern
(e.g. "I have {int} cucumbers") alongside ~r/regex/, routed through the
existing Cabbage.CucumberExpression engine. Arguments arrive as a positional
list of transformed values ({int} -> integer, {float} -> float,
{string}/{word} -> string), distinct from the regex path's named-captures map,
which is unchanged. The removed {name:type} sugar still raises the engine's
"Undefined parameter type" compile error at the definition site.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
defstep/4, defand/4 and defbut/4 delegate to Helpers.add_step/6 exactly as defgiven/defwhen/defthen, with their own metadata function tag. Matching is already keyword-agnostic, so these register identical 5-tuples and match by pattern only. Every step macro also gets a /3 short form (pattern, state, do: block) whose vars defaults to a hygienic ignore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cabbage.Steps is a lightweight step library: it registers the accumulating @steps/@tags attributes, imports the full Cabbage.Feature macro surface, and reuses Cabbage.Feature.expose_metadata/1 to emit raw_steps/0, raw_tags/0 and __cabbage_document__/0 (nil). It does not use ExUnit.Case and generates no tests, so it is a drop-in import source. Cabbage.Feature gains an :import option that expands each module to import_steps/import_tags during use. Imported steps accumulate into a dedicated @imported_steps attribute and are appended after the importer's own @steps, so first-match-wins keeps a local step ahead of a same-pattern imported one. The imported ordering is deterministic: source-definition order within each module, modules in import order. import_steps now raises a clear error when given a module that is not compiled or does not export raw_steps/0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inject :__table__ and :__doc_string__ into the context each step sees, sourced from the step's gherkin data and re-injected per step rather than threaded. This gives cucumber-expression steps (positional vars, no room for named :table/:doc_string) a uniform way to read them. The reserved keys are stripped from the threaded state on the way out and added to remove_hidden_state/1's drop list so they never leak into a user's state-pattern destructure. No change to the return-value contract. Also fix the long-standing `case_templae` typo in the hidden-keys list; :case_template is now actually stripped, so the template test observes its CaseTemplate via a non-reserved key instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
T6 will make a step's final return drive the context explicitly and raise
on non-conforming returns. Today the suite leans on the silent-keep escape
hatch: many `defthen` blocks end in a bare `assert` (returns a truthy
boolean, currently discarded). Make every executable cabbage step return a
conforming value (`:ok`/`nil`/map/`{:ok, map}`) so it is correct under both
the old and new contracts, with no behaviour change here.
Sites fixed (final expression relied on silent-keep):
- feature_execution_test.exs: the `[:given | state]` keep-state step now
returns :ok explicitly; eight `assert`-final Given/Then steps get :ok
- feature_state_threading_test.exs: four `assert`-final steps
- feature_import_test.exs: three `assert`-final Then steps
- feature_setup_test.exs: two `assert`-final Then steps
- feature_steps_module_test.exs: two `assert`-final Then steps
- feature_defstep_test.exs, feature_template_test.exs: one each
- formatter_test.exs: three `do: assert(true)` one-liners become
block-form steps ending in :ok
Add test/feature_state_contract_test.exs covering the full target contract
(:ok/nil keep, {:ok, map} merge, bare-map replace, {:error, _}/other/struct
raise), tagged :skip until T6 flips the contract and un-skips it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the silent `{:ok, map} | _ -> keep` rule with the explicit D3
contract via Helpers.apply_step_return/5:
:ok | nil -> context unchanged
{:ok, map} -> Map.merge(context, map) (delta-merge; back-compat)
a bare map -> replaces the context
{:error, reason} -> raises, naming the step and reason
a struct -> raises (context must be a plain map)
anything else -> raises, naming the step and the offending value
Clause order is load-bearing: `{:error, _}` is a tuple and a struct
satisfies is_map/1, so the struct clauses (bare and `{:ok, _}`-wrapped)
are matched before their plain-map counterparts. A struct return raises
rather than replacing the context, because a struct context would break
the map-pattern destructuring used to bind state; `{:ok, struct}` is
rejected the same way so struct fields (and :__struct__) can never merge
in through the wrapper. Failures reraise with the step's own metadata for
a useful stacktrace.
Update the moduledoc's "Modifying State" section to the full contract
(merge vs replace vs keep vs raise, struct rejection in both forms) plus
the :__table__/:__doc_string__ reserved keys, fix the moduledoc examples
to return :ok, and un-skip test/feature_state_contract_test.exs. The
"error on returning {:ok, not a map}" execution test now asserts the new
clear error instead of BadMapError.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the binary-with-{...}/literal/regex classification (and the {name:type}
validation UX) into a shared, dependency-free module. Feature-layer
to_pattern_ast/1 and eval_pattern/1 delegate; AST storage stays in helpers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move step_matches?/extract_named_vars into a shared match/2 returning nil |
{:ok, named_captures} (regex, string-keyed) | {:ok, positional_list}
(expression). A successful zero-capture match stays {:ok, _}, not nil. The
Feature layer atomizes the regex capture keys at its own surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
StepRegistry.build_definition/3 now classifies via the shared module: classify_expression/2 compiles a binary as a Cucumber Expression (Messages policy: every binary is an expression, even a braceless one) and passes a Regex through verbatim. The undefined-parameter-type reporting path and the binary-vs-regex dispatch stay in StepRegistry; Matcher offset computation is untouched. Conformance remains 0 failures / 1 skipped (byte-identical). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrite the user-facing docs to describe the final 1.0.0 step API as a single
coherent narrative, with no intermediate-state phrasing:
- Cabbage.Feature moduledoc: lead with a realistic feature file + shared
Cabbage.Steps module + per-feature module; state matching is keyword-agnostic
(defstep canonical, defgiven/defwhen/defthen/defand/defbut aliases); lead step
patterns with the {int} Cucumber Expression before the regex form; document the
D3 state contract table (keep/merge/replace/raise) and ctx.__table__ /
ctx.__doc_string__; replace the file-less-module "Organizing Features" section
with "Sharing steps with Cabbage.Steps" + import:.
- README: rewrite Example Usage around a shared steps module + import, leading
with the Cucumber Expression step; add the keyword-agnostic note and the state
contract table; fix Tables & Doc Strings to ctx.__table__/ctx.__doc_string__.
- UPGRADING.md: soften the "code does not change" headline; add keyword-agnostic,
state-contract (silent-keep removed, remediation diff, bare-map replace vs
{:ok,map} merge), and Cabbage.Steps sharing sections; keep {name:type} and
valued-tag sections.
- CHANGELOG.md: fold D1-D4 under the single 1.0.0 heading; add the state-contract
breaking note, keyword-agnostic macros, Cabbage.Steps, reserved context keys,
and the shared StepPattern core.
Also annotate two opaque types (Cabbage.TagExpression.t, CucumberExpressionError.t)
so `mix docs` builds with zero warnings (the docs gate).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Any news on that? I checked the version from Tomasz and it just works. Would be nice to see it merged... |
Collaborator
Author
|
@arfl thanks for the interest! When I made this, I did reach out to the maintainers of the projects and they gave me ownership access over both, so I can merge it, I was just slightly hesitant to and hoped to have someone test it etc. that isn't me so appreciate you checking it out! At the same time, I obviously steered CC to vibe code all of this so I'm not 100% convinced about future maintainability - with that said, the current maintainers moved on as well, so not sure if this is better or worse 🤷 In the process of working on this I also discovered https://github.com/huddlz-hq/cucumber which is human-maintained and the wonderful @mwoods79 took the main thing I saw cabbage missing, the CCK compatibility and shipped it with 1.0.0 release there recently. IDK if it's worth continuing with these PRs - perhaps you could try the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello! I'm Tomasz, I've contacted the maintainer of this GH org with the interest of updating both
gherkinandcabbagepackages to the latest & greatest standards. I've driven Claude through this, as it is indeed a lot of work, but thankfully the CCK from Cucumber makes this a great candidate for LLM tools IMHO!What this is
A proposal to revive and modernize
cabbageas the canonical Elixir Cucumber runner, built on the companion gherkin 3.0 rewrite (cabbage-ex/gherkin#23). This is a near-total rewrite — please review as a proposal with evidence, not a line-by-line diff. It targetsnext(notmaster).Highlights
Cabbage.Featurestep macros —defgiven "I have {int} cucumbers", [count], state— with typed positional args ({int}→ integer,{float}→ float), not only in theCabbage.Messageslayer. Regex patterns are unchanged.Cabbage.Feature(compile-time macros → ExUnit tests, what users write) andCabbage.Messages(runtime cucumber-messages interpreter, what the CCK grades).Cabbage.Formatterbridges them — an ExUnit formatter that emits cucumber-messages frommix test.setup_tag, group-by-feature, untagged scenarios, ambiguous-step detection.defstepis the canonical macro;defgiven/defwhen/defthen/defand/defbutare readability aliases (each in /3 and /4 arities). Matching is by pattern only, per the Cucumber spec — keyword is irrelevant at match time.use Cabbage.Stepsmodule and pull them in withuse Cabbage.Feature, file: ..., import: [MySteps]. Local steps win on first-match; import order is deterministic.:ok/nilpreserves context;{:ok, map}merges; a bare map replaces. Structs,{:error, _}, and any other return value raise immediately — the old silent-keep behaviour is removed. Reserved per-step keys::__table__and:__doc_string__. SeeUPGRADING.md.Cabbage.StepPatterncore — pattern classify/match logic is extracted into a single module used by both theCabbage.Featuremacro layer and theCabbage.Messagesruntime interpreter;Messagesbehaviour is byte-identical to before.Evidence
mix compile --warnings-as-errorsclean ·mix test→ 2 doctests, 246 tests, 0 failures ·mix test --only conformance→ 0 failures, 1 skipped (expected) ·mix format --check-formattedclean.{name:type}step sugar removed — was never part of the Cucumber Expressions spec; such patterns now raise a clear compile-time error. Migrate{count:int}→{int}(Cucumber Expression, typed), or~r/(?<count>\d+)/when a named binding is wanted.:ok/nil(keep),{:ok, map}(merge), and a bare map (replace) are valid. Any other return (structs,{:error, _}, etc.) raises immediately.See
UPGRADING.mdfor migration guidance on both points.Standing issues / PRs this addresses
Addressed by the rewrite: #47 cucumber expressions · #53 step logging · #61 broken README links · #64 exact-string match · #68 background · #69 Rule · #71 setup/setup_all · #73 group by feature · #74 tag→setup_tag · #75 version matrix · #77 single import path · #83 But keyword · #88 ambiguous steps · #93 deprecation warnings · #94 untagged scenarios.
Resolved differently: #65 keyword-list result — superseded by the explicit state contract (non-map returns now raise; see
UPGRADING.md).Obsoleted PRs: #97 (deprecation/version fixes — rewrite is warnings-clean 1.18–1.20) · #89 (test-name duplication — generation rewritten) · #85 (typo/formatting — README rewritten).
Deferred / optional: #72 rename to
Cabbage.Case(cosmetic, 2.0) · #5 OSS icon (not parity).Notes
1.0.0(never published to Hex).