dav1d crashes generator

[DemiMarie]

[246/359] Generating interface
FAILED: contrib/dav1d/dav1d_sapi.sapi.h contrib/dav1d/dav1d_sapi.isystem /home/user/sandboxed-api/build/contrib/dav1d/dav1d_sapi.sapi.h /home/user/sandboxed-api/build/contrib/dav1d/dav1d_sapi.isystem 
cd /home/user/sandboxed-api/build/contrib/dav1d && sh -c "/usr/bin/clang++ -E -x c++ -v /dev/null 2>&1 |              awk '/> search starts here:/{f=1;next}/^End of search/{f=0}f{print \$1}'              > \"dav1d_sapi.isystem\"" && /usr/bin/python3.10 -B /home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py --sapi_name=dav1d --sapi_out=dav1d_sapi.sapi.h --sapi_embed_dir=/home/user/sandboxed-api/build/contrib/dav1d --sapi_embed_name=dav1d_sapi --sapi_functions= --sapi_ns=dav1d_sapi --sapi_isystem=dav1d_sapi.isystem --sapi_in=/usr/include/dav1d/dav1d.h
Traceback (most recent call last):
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py", line 73, in <module>
    app.run(main)
  File "/usr/lib/python3.10/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/usr/lib/python3.10/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py", line 60, in main
    result = generator.generate(FLAGS.sapi_name, FLAGS.sapi_functions,
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 729, in generate
    related_types = self._get_related_types(function_names)
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 790, in _get_related_types
    types += sorted(r for r in fn_related_types if r not in processed)
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 145, in __lt__
    return (self._tu.order[self._get_declaration().hash] <
KeyError: 2210228446
[249/359] Generating interface
FAILED: contrib/c-ares/cares_sapi.sapi.h contrib/c-ares/cares_sapi.isystem /home/user/sandboxed-api/build/contrib/c-ares/cares_sapi.sapi.h /home/user/sandboxed-api/build/contrib/c-ares/cares_sapi.isystem 
cd /home/user/sandboxed-api/build/contrib/c-ares && sh -c "/usr/bin/clang++ -E -x c++ -v /dev/null 2>&1 |              awk '/> search starts here:/{f=1;next}/^End of search/{f=0}f{print \$1}'              > \"cares_sapi.isystem\"" && /usr/bin/python3.10 -B /home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py --sapi_name=cares --sapi_out=cares_sapi.sapi.h --sapi_embed_dir=/home/user/sandboxed-api/build/contrib/c-ares --sapi_embed_name=cares_sapi --sapi_functions= --sapi_ns=c_ares_sapi --sapi_isystem=cares_sapi.isystem --sapi_in=/usr/include/ares.h
Traceback (most recent call last):
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py", line 73, in <module>
    app.run(main)
  File "/usr/lib/python3.10/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/usr/lib/python3.10/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/sapi_generator.py", line 60, in main
    result = generator.generate(FLAGS.sapi_name, FLAGS.sapi_functions,
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 744, in generate
    return self.format_template(**api)
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 945, in format_template
    result.append(self._format_function(f))
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 856, in _format_function
    argument_types.append(a.wrapped + ';')
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 412, in wrapped
    return '{} {name}_(({name}))'.format(self.mapped_type, name=self.name)
  File "/home/user/sandboxed-api/sandboxed_api/tools/generator2/code.py", line 443, in mapped_type
    raise ValueError('Elaborate type (eg. struct) in mapped_type is not '
ValueError: Elaborate type (eg. struct) in mapped_type is not supported: function getipv4sourcefilter, arg 1, type struct in_addr, location <SourceLocation file '/usr/include/netinet/in.h', line 607, column 12>
[251/359] Building CXX object sandboxed_api/examples/zlib/CMakeFiles/sapi_main_zlib.dir/main_zlib.cc.o
ninja: build stopped: subcommand failed.

[cblichmann]

Is this the function signature?

int getipv4sourcefilter(int s, struct in_addr interface, struct in_addr group, uint32_t *fmode,
                        uint32_t *numsrc, struct in_addr *slist);

At least for c-ares, this error is not a crash but something we deliberately do not support (elaborate types). The generator does not know what to do with the interface and group parameters.

[DemiMarie]

Is this the function signature?

int getipv4sourcefilter(int s, struct in_addr interface, struct in_addr group, uint32_t *fmode,
                        uint32_t *numsrc, struct in_addr *slist);

At least for c-ares, this error is not a crash but something we deliberately do not support (elaborate types). The generator does not know what to do with the interface and group parameters.

Interesting, what is the reason for this? Can it be worked around by writing a wrapper function? Also, an uncaught exception is not the best way to report an error.

The KeyError is definitely not expected. What is the cause of that?

[cblichmann]

Is this the function signature?

int getipv4sourcefilter(int s, struct in_addr interface, struct in_addr group, uint32_t *fmode,
                        uint32_t *numsrc, struct in_addr *slist);

At least for c-ares, this error is not a crash but something we deliberately do not support (elaborate types). The generator does not know what to do with the interface and group parameters.

Interesting, what is the reason for this? Can it be worked around by writing a wrapper function?

A wrapper function will work. For struct in_addr we can probably just cast it into uint32_t.
The reason for this is that we treat structs as C++ types, which are basically classes. And passing around classes means we have name mangling and a lot more behavior going on than in C.

For the simple call-by-value case, we should enhance the generator to support this.

Also, an uncaught exception is not the best way to report an error.

Yes of course. Then again, there is a lot of code at Google that hard-fails on any error. Usually that means with a long-ish stack trace.

The KeyError is definitely not expected. What is the cause of that?

I can't tell right now, need to debug this.

[DemiMarie]

Is this the function signature?

int getipv4sourcefilter(int s, struct in_addr interface, struct in_addr group, uint32_t *fmode,
                        uint32_t *numsrc, struct in_addr *slist);

At least for c-ares, this error is not a crash but something we deliberately do not support (elaborate types). The generator does not know what to do with the interface and group parameters.

Interesting, what is the reason for this? Can it be worked around by writing a wrapper function?

A wrapper function will work. For struct in_addr we can probably just cast it into uint32_t. The reason for this is that we treat structs as C++ types, which are basically classes. And passing around classes means we have name mangling and a lot more behavior going on than in C.

For the simple call-by-value case, we should enhance the generator to support this.

👍, though this might also be a motivation to finish the libtooling-based generator.

Also, an uncaught exception is not the best way to report an error.

Yes of course. Then again, there is a lot of code at Google that hard-fails on any error. Usually that means with a long-ish stack trace.

There is code in Qubes OS that also has poor error reporting, so I understand.

The KeyError is definitely not expected. What is the cause of that?

I can't tell right now, need to debug this.

Thanks