Skip to content

Don't load rules from an unsupported standard version - #529

Merged
nunoplopes merged 1 commit into
Cpp2Rust:masterfrom
lucic71:standard-ifdefs
Oct 14, 2026
Merged

Don't load rules from an unsupported standard version#529
nunoplopes merged 1 commit into
Cpp2Rust:masterfrom
lucic71:standard-ifdefs

Conversation

@lucic71

@lucic71 lucic71 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

In rules/, standard specific rules are guarded by #if __cplusplus >= 202002L. We will also have cases where a rule is only available up to a specific standard version, then deprecated.

This adds a new attribute in the rules index: supported standards. The loader only loads rules which have the same standard as the current TU.

@nunoplopes

nunoplopes commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Seems weird to crawl the rules a dozen times. Can't the indexer itself parse the same file multiple times instead of calling the indexer many times and producing many times.
An index can contain all versions.
Why are we even parsing; a regex could do the same thing in one go..

@lucic71

lucic71 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

Seems weird to crawl the rules a dozen times. Can't the indexer itself parse the same file multiple times instead of calling the indexer many times and producing many times.

I did this

Why are we even parsing; a regex could do the same thing in one go..

We need parsing to get declarations. Declarations can change between versions. For example emplace_back returns void in C++11 and reference type in C++17.

On the other side, if the declaration does not change between versions, with parsing we catch errors like: the rule author wrote a C++11 rule without guarding it.

@lucic71

lucic71 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

Seems weird to crawl the rules a dozen times. Can't the indexer itself parse the same file multiple times instead of calling the indexer many times and producing many times.

I did this

Why are we even parsing; a regex could do the same thing in one go..

We need parsing to get declarations. Declarations can change between versions. For example emplace_back returns void in C++11 and reference type in C++17.

On the other side, if the declaration does not change between versions, with parsing we catch errors like: the rule author wrote a C++11 rule without guarding it.

I think a cheaper solution is: split src.cpp in multiple files based on standard like src_cxx11.cpp, src_cxx20.cpp. This way we know exactly what is the standard version a file should be parsed with. @nunoplopes what do you think?

@lucic71 lucic71 mentioned this pull request Oct 12, 2026
Merged
@nunoplopes
nunoplopes merged commit d84269b into Cpp2Rust:master Oct 14, 2026
9 checks passed
nunoplopes pushed a commit that referenced this pull request Oct 14, 2026
@lucic71
Replace string matcher with AST matcher (#527)
e65e265 
Closes #447, depends on #529

Rules are matched on the AST instead of on printed strings.
string_matcher and ir_src.json are deleted. cpp-rule-prepocessor is
replaced by cpp-rule-indexer which walks the rules, compiles them, and
creates an index (#451).

---

The bodies of the used rules are loaded in the TU to match against their
usages.

A rule matches a usage if calling the rule with the operands of the
usage would end up calling the same declaration as the usage.
Concretely:

1. The usages is keyed by its declaration: `Member{class, member_name}`,
`Constructor{class}`, `Unqualified{name}`. In Member we keep: data
members, static members, function members. In Unqualified we keep
everything outside a class: free function, macro, enumerators, etc.
Every loaded rule with the same key is a candidate.
2. Between a usage and a candidate rule, let clang deduce the template
arguments of a rule by calling the rule with the usage's operands. If
this does not work (T1, T2, not used in the argument list), deduce using
the return type.
3. The deduced arguments are subtituted in the rule's return expression.
If the resolved expressoin has the same declaration as the usage, then
it's a match.
4. If several rules mtach, non-template rules win over template rules.
Otherwise let clang's partial ordering pick the most specialized one

For example:

```cpp
std::vector<int> v;
v.push_back(3);

template <typename T1> void f1(std::vector<T1> &a0, const T1& a1) {
  return a0.push_back(a1);
}

template <typename T1> void f2(std::vector<T1> &a0, T1&& a1) {
  return a0.push_back(std::move(a1));
}
```
At load time: both f1 and f2 are stored under `Member{vector,
push_back}`.

1. Mapper calls `RuleRegistry::Search(expr)` which reaches
`RuleRegistry::findExpr`. The decl of the usage
(`vector<int>::push_back(int&&)`) is also keyed under `Member{vector,
push_back}`.
2. `Match` takes the operands `[v, 3]` from the usage and gets:
  i.  `f1(vector<T1>&, const T1&)` -> `T1 = int`
  ii. `f2(vector<T1>&, T1&&)` -> `T1 = int`
3. The deduced tempalate arguments are inserted in the return
expression:
i. f1: `a0.push_back(a1)` calls `push_back(const int&)`. Does not match
the usage decl.
ii. f2: `a0.push_back(std::move(a1))` calls `push_back(int&&)`. It
matches the usage decl.
4. Only f2 is matches so there is no need for disambiguation. 

Type rules work the same. The usage type is deduced against the rule's
alias with sugar preferred over the canonical type.

---

Not all rules have deducible arguments, for example `std::map<T1,
T2>::const_iterator`, in general every template argument that sits on
the left of `::` is non-deducible. Such rules need changes so they
become deducible:

```diff
- template <typename T1, typename T2>
- using t2 = typename std::map<T1, T2>::const_iterator;
+ template <typename It, typename T1 = KeyOf<It>, typename T2 = MappedOf<It>,
+          typename = SameAs<It, typename std::map<T1, T2>::const_iterator>>
+ using t2 = It;
```

Both T1 and T2 originally sat in non-deducible contexts. The new syntax
works like this:
1. The iterator is taken as plain `It` which is always deducible
2. `T1` and `T2` are computed from `It` using iterator traits: `T1 =
It::value_type::first_type` and `T2 = It::value_type::second_type`
3. `SameAs` constraints the rule only for const iterators of `std::map`
so that they don't conflict with the rules for non-cosnt iterators for
`std::map` or other map types

---

Performance

| branch | unit (500 TUs) | woff2 (20 TUs) | brunsli (17 TUs) |
description |
| --- | --- | --- | --- | --- | 
| master (unsafe + refcount) (s) | 33.2 | 11.9 | 26.5 | |
| ast-matcher (unsafe + refcount) (s) | 32.7 | 11.6 | 25.6 | |
| **speedup** per TU (ms)| 0.5 | 7.5 | 26.5 | (master - ast-matcher) / 2
/ number-of-TUs * 1000 |
| total **speedup** (%) | 1.5 | 2.5 | 3.4 | (master - ast-matcher) /
master * 100 |

---

Some nice implications for rule writing:
1. deque_cxx11 and vector_cxx11 disappear. Instead we change the
definition of functions:

```cpp
auto f12(std::deque<T1> &o, Init<T1, Args> &&...args)
    // in C++11 the return type is deduced as void
    // post-C++11 the return type is deduced as reference type
    -> decltype(o.emplace_back(std::forward<Args>(args)...)) {
  return o.emplace_back(std::forward<Args>(args)...);
}
```

2. We don't need to write rules for `T*`, `T**`, `T&`, `const T*`, etc.
We only write one rule for T and the converter takes care of the rest,
see `rules/builtin_types/src.c` and `rules/iostream/src.cpp`
3. We only write one constructor for std::pair that forwards the
arguments, see `rules/pair/src.cpp`
4. We only write rules for `vector<T, A>`, rules for `vector<T>` are not
needed. This drops a lot of useless rules in `rules/vector/src.cpp`

This opens another new interesting opportunity: writing rules for
multiple statements at once. Now that we match AST subtrees we can match
an entire subtree at once and write more idiomatic rules. Also we can
move some of the converter translations in rules: for exmaple binary
operators which have a very complicated visitor today.

---

Next steps after this PR:
1. The AST type matcher stops walking the sugar typedefs and starts
translating rules for each typedef: off_t, value_type, time_t, etc
2. Stop masking NTTPs in rules (#435)
3. Add a cache on top of mapper to avoid deduction for results we
already mapped
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lucic71 @nunoplopes