Use Python to combine verification output#16
Conversation
AndresYague
left a comment
There was a problem hiding this comment.
Simple MR that makes the code more portable as it does not require that the system has cat. The code will be slower than running cat but not by a significant enough margin.
It may be useful to provide a buffer length of a few MB to shutil.copyfileobj to handle the 3D case output a bit faster.
Thanks, I added a 1 MiB buffer. In a local 128 MiB 3D-shaped merge, it was 1.34x faster than a 64 KiB buffer. The output test and docs check pass. |
This all looks good to me! |
Change
find_last_outputusedcatto merge rank CSV files. This now uses Python file I/O, with the existing output order unchanged.Checks
python tools/test_verify_output.pyruff check toolsgit diff --checkCMake and C++ unit tests were not run locally because CMake and FleCSI are not installed here.
Closes #15