Skip to content

Add AIX support#1065

Open
KV2773 wants to merge 3 commits into
microsoft:mainfrom
KV2773:aix-platform
Open

Add AIX support#1065
KV2773 wants to merge 3 commits into
microsoft:mainfrom
KV2773:aix-platform

Conversation

@KV2773

@KV2773 KV2773 commented May 20, 2026

Copy link
Copy Markdown

Patches for building ort-extension on AIX, related to big endianness and some aix specific headers.

Copilot AI review requested due to automatic review settings May 20, 2026 08:53
@KV2773 KV2773 requested a review from a team as a code owner May 20, 2026 08:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds IBM AIX build support across the core shared library build and several bundled third-party dependencies, including handling of AIX-specific macro collisions and big-endian data parsing.

Changes:

  • Add AIX-specific exported-symbol list support for the shared library and wire it into the build.
  • Patch external dependencies (SentencePiece, OpenCV/libpng, dlib, dr_libs, BlingFire) to compile on AIX.
  • Update the UGM tokenizer’s precompiled charsmap parsing to avoid strict-aliasing and handle big-endian hosts.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
shared/ortcustomops.exp Adds an AIX export list for shared-library symbol exports.
shared/api/nemo_mel_spectrogram.h Undefines hz on AIX to avoid macro collisions with parameter names.
operators/tokenizer/ugm_kernels.hpp Uses memcpy for header reads and swaps bytes on big-endian systems.
operators/text/vector_to_string.hpp Renames the std::hash parameter (no functional change).
CMakeLists.txt Uses .exp exports on AIX and .ver version scripts on Linux/Android.
cmake/externals/sentencepieceproject.cmake Adds AIX/enum patches to the SentencePiece patch set.
cmake/externals/sentencepieceproject_enum.patch Forces ScriptType enum underlying type to int32_t.
cmake/externals/sentencepieceproject_aix.patch Defines endianness macro handling for AIX in SentencePiece.
cmake/externals/rand_aix.patch Adds AIX fixes to OpenCV sources (macro collision + ELF/auxv guards).
cmake/externals/opencv.cmake Applies additional AIX-specific OpenCV patches during fetch.
cmake/externals/opencv_libpng_aix.patch Enables libpng PowerPC VSX optimizations on AIX.
cmake/externals/dr_libs.cmake Applies an AIX patch to dr_libs during fetch.
cmake/externals/dr_libs_aix.patch Undefines hz on AIX for dr_mp3.
cmake/externals/dlib.cmake Applies an AIX patch to dlib during fetch.
cmake/externals/dlib_aix.patch Updates dlib’s embedded libpng VSX detection to include AIX.
cmake/externals/blingfire.cmake Applies an AIX patch and links Iconv on AIX.
cmake/externals/blingfire-aix.patch Adjusts BlingFire headers for AIX (but currently breaks macOS).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread operators/tokenizer/ugm_kernels.hpp Outdated
#include <unordered_map>
#include <cwctype>
#include <locale>
#include <cstring>
Comment on lines +98 to +99
uint32_t xcda_blob_size;
std::memcpy(&xcda_blob_size, &charsmap_data_[0], sizeof(xcda_blob_size));
Comment on lines 13 to 17

template <class T>
struct hash<std::vector<T>> {
size_t operator()(const vector<T>& __vector) const noexcept;
size_t operator()(const vector<T>& __vec) const noexcept;
};
Comment thread cmake/externals/blingfire.cmake Outdated
Comment on lines 11 to 14
link_libraries(Iconv::Iconv)
endif()

FetchContent_MakeAvailable(Blingfire)
Comment thread cmake/externals/blingfire-aix.patch Outdated

@sayanshaw24 sayanshaw24 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CIs are still failing due to a corrupt patch, please try to repro by building locally and ensure all errors are fixed:

  Building Custom Rule E:/_work/1/s/out/Windows/_deps/dr_libs-subbuild/CMakeLists.txt
  No update step for 'dr_libs-populate'
  Performing patch step for 'dr_libs-populate'
CUSTOMBUILD : error : corrupt patch at line 6 [E:\_work\1\s\out\Windows\_deps\dr_libs-subbuild\dr_libs-populate.vcxproj]

@KV2773

KV2773 commented Jun 2, 2026

Copy link
Copy Markdown
Author

CIs are still failing due to a corrupt patch, please try to repro by building locally and ensure all errors are fixed:

  Building Custom Rule E:/_work/1/s/out/Windows/_deps/dr_libs-subbuild/CMakeLists.txt
  No update step for 'dr_libs-populate'
  Performing patch step for 'dr_libs-populate'
CUSTOMBUILD : error : corrupt patch at line 6 [E:\_work\1\s\out\Windows\_deps\dr_libs-subbuild\dr_libs-populate.vcxproj]

Hii @sayanshaw24 is it possible for the community to add a .gitattribute file similar to
https://github.com/microsoft/onnxruntime/blob/main/cmake/patches/.gitattributes
in cmake/externals/ folder
as I have tested by adding it on AIX,Linux, and Windows, it works fine, as making all platforms to use LF format for line endings.

@KV2773 KV2773 requested review from Copilot and sayanshaw24 June 4, 2026 03:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

Comment thread operators/tokenizer/ugm_kernels.hpp
Comment thread operators/tokenizer/ugm_kernels.hpp
Comment thread operators/tokenizer/ugm_kernels.hpp
Comment thread shared/api/nemo_mel_spectrogram.h
Comment thread operators/text/vector_to_string.hpp
Comment thread cmake/externals/blingfire-aix.patch
KV2773 and others added 3 commits June 4, 2026 08:58
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@KV2773

KV2773 commented Jun 9, 2026

Copy link
Copy Markdown
Author

Hii @sayanshaw24 , may you review it once.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants