Add AIX support#1065
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds IBM AIX build support across the core shared library build and several bundled third-party dependencies, including handling of AIX-specific macro collisions and big-endian data parsing.
Changes:
- Add AIX-specific exported-symbol list support for the shared library and wire it into the build.
- Patch external dependencies (SentencePiece, OpenCV/libpng, dlib, dr_libs, BlingFire) to compile on AIX.
- Update the UGM tokenizer’s precompiled charsmap parsing to avoid strict-aliasing and handle big-endian hosts.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| shared/ortcustomops.exp | Adds an AIX export list for shared-library symbol exports. |
| shared/api/nemo_mel_spectrogram.h | Undefines hz on AIX to avoid macro collisions with parameter names. |
| operators/tokenizer/ugm_kernels.hpp | Uses memcpy for header reads and swaps bytes on big-endian systems. |
| operators/text/vector_to_string.hpp | Renames the std::hash parameter (no functional change). |
| CMakeLists.txt | Uses .exp exports on AIX and .ver version scripts on Linux/Android. |
| cmake/externals/sentencepieceproject.cmake | Adds AIX/enum patches to the SentencePiece patch set. |
| cmake/externals/sentencepieceproject_enum.patch | Forces ScriptType enum underlying type to int32_t. |
| cmake/externals/sentencepieceproject_aix.patch | Defines endianness macro handling for AIX in SentencePiece. |
| cmake/externals/rand_aix.patch | Adds AIX fixes to OpenCV sources (macro collision + ELF/auxv guards). |
| cmake/externals/opencv.cmake | Applies additional AIX-specific OpenCV patches during fetch. |
| cmake/externals/opencv_libpng_aix.patch | Enables libpng PowerPC VSX optimizations on AIX. |
| cmake/externals/dr_libs.cmake | Applies an AIX patch to dr_libs during fetch. |
| cmake/externals/dr_libs_aix.patch | Undefines hz on AIX for dr_mp3. |
| cmake/externals/dlib.cmake | Applies an AIX patch to dlib during fetch. |
| cmake/externals/dlib_aix.patch | Updates dlib’s embedded libpng VSX detection to include AIX. |
| cmake/externals/blingfire.cmake | Applies an AIX patch and links Iconv on AIX. |
| cmake/externals/blingfire-aix.patch | Adjusts BlingFire headers for AIX (but currently breaks macOS). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #include <unordered_map> | ||
| #include <cwctype> | ||
| #include <locale> | ||
| #include <cstring> |
| uint32_t xcda_blob_size; | ||
| std::memcpy(&xcda_blob_size, &charsmap_data_[0], sizeof(xcda_blob_size)); |
|
|
||
| template <class T> | ||
| struct hash<std::vector<T>> { | ||
| size_t operator()(const vector<T>& __vector) const noexcept; | ||
| size_t operator()(const vector<T>& __vec) const noexcept; | ||
| }; |
| link_libraries(Iconv::Iconv) | ||
| endif() | ||
|
|
||
| FetchContent_MakeAvailable(Blingfire) |
sayanshaw24
left a comment
There was a problem hiding this comment.
CIs are still failing due to a corrupt patch, please try to repro by building locally and ensure all errors are fixed:
Building Custom Rule E:/_work/1/s/out/Windows/_deps/dr_libs-subbuild/CMakeLists.txt
No update step for 'dr_libs-populate'
Performing patch step for 'dr_libs-populate'
CUSTOMBUILD : error : corrupt patch at line 6 [E:\_work\1\s\out\Windows\_deps\dr_libs-subbuild\dr_libs-populate.vcxproj]
Hii @sayanshaw24 is it possible for the community to add a .gitattribute file similar to |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Hii @sayanshaw24 , may you review it once. |
Patches for building ort-extension on AIX, related to big endianness and some aix specific headers.