Elixir bindings to the XGBoost C API using Native Implemented Functions (NIFs).
EXGBoost provides an implementation of XGBoost that works with
Nx tensors.
Xtreme Gradient Boosting (XGBoost) is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.
def deps do
[
{:exgboost, "~> 0.6"}
]
endEXGBoost's top-level EXGBoost API works directly and only with Nx tensors. However, under the hood,
it leverages the structs defined in the EXGBoost.Booster and EXGBoost.DMatrix modules. These structs
are wrappers around the structs defined in the XGBoost library.
The two main structs used are DMatrix
to represent the data matrix that will be used
to train the model, and Booster
which represents the model.
The top-level EXGBoost API does not expose the structs directly. Instead, the
structs are exposed through the EXGBoost.Booster and EXGBoost.DMatrix modules. Power users
might wish to use these modules directly. For example, if you wish to use the Booster struct
directly then you can use the EXGBoost.Booster.booster/2 function to create a Booster struct
from a DMatrix and a keyword list of options. See the EXGBoost.Booster and EXGBoost.DMatrix
modules source for more implementation details.
key = Nx.Random.key(42)
{x, key} = Nx.Random.normal(key, 0, 1, shape: {10, 5})
{y, key} = Nx.Random.normal(key, 0, 1, shape: {10})
model = EXGBoost.train(x, y)
EXGBoost.predict(model, x)EXGBoost is designed to feel familiar to the users of the Python XGBoost library. EXGBoost.train/2 is the
primary entry point for training a model. It accepts a Nx tensor for the features and a Nx tensor for the labels.
EXGBoost.train/2 returns a trained Booster struct that can be used for prediction. EXGBoost.train/2 also
accepts a keyword list of options that can be used to configure the training process. See the
XGBoost documentation for the full list of options.
EXGBoost.train/2 uses the EXGBoost.Training.train/2 function to perform the actual training. EXGBoost.Training.train/2
and can be used directly if you wish to work directly with the DMatrix and Booster structs.
One of the main features of EXGBoost.train/2 is the ability for the end user to provide a custom training function
that will be used to train the model. This is done by passing a function to the :obj option. The function must
accept a DMatrix and a Booster and return a Booster. The function will be called at each iteration of the
training process. This allows the user to implement custom training logic. For example, the user could implement
a custom loss function or a custom metric function. See the XGBoost documentation
for more information on custom loss functions and custom metric functions.
Another feature of EXGBoost.train/2 is the ability to provide a validation set for early stopping. This is done
by passing a list of 3-tuples to the :evals option. Each 3-tuple should contain a Nx tensor for the features, a Nx tensor
for the labels, and a string label for the validation set name. The validation set will be used to calculate the validation
error at each iteration of the training process. If the validation error does not improve for :early_stopping_rounds iterations
then the training process will stop. See the XGBoost documentation
for a more detailed explanation of early stopping.
Early stopping is achieved through the use of callbacks. EXGBoost.train/2 accepts a list of callbacks that will be called
at each iteration of the training process. The callbacks can be used to implement custom logic. For example, the user could
implement a callback that will print the validation error at each iteration of the training process or to provide a custom
setup function for training. See the EXGBoost.Training.Callback module for more information on callbacks.
Please notes that callbacks are called in the order that they are provided. If you provide multiple callbacks that modify
the same parameter then the last callback will trump the previous callbacks. For example, if you provide a callback that
sets the :early_stopping_rounds parameter to 10 and then provide a callback that sets the :early_stopping_rounds parameter
to 20 then the :early_stopping_rounds parameter will be set to 20.
You are also able to pass parameters to be applied to the Booster model using the :params option. These parameters will
be applied to the Booster model before training begins. This allows you to set parameters that are not available as options
to EXGBoost.train/2. See the XGBoost documentation for a full
list of parameters.
EXGBoost.train(X,
y,
evals: [{X_test, y_test, "test"}],
learning_rates: fn i -> i/10 end,
num_boost_rounds: 10,
early_stopping_rounds: 3,
max_depth: 3,
eval_metric: [:rmse,:logloss]
)EXGBoost.predict/2 is the primary entry point for making predictions with a trained model.
It accepts a Booster struct (which is the output of EXGBoost.train/2).
EXGBoost.predict/2 returns a Nx tensor containing the predictions.
EXGBoost.predict/2 also accepts a keyword list of options that can be used to configure the prediction process.
preds = EXGBoost.train(X, y) |> EXGBoost.predict(X)Important: Booster objects are not thread-safe for concurrent predictions. The underlying XGBoost C API does not provide synchronization mechanisms, and sharing a single booster reference across multiple Elixir processes for concurrent predictions can lead to race conditions, memory corruption, or incorrect results. For this reason it is not recommended that you cache boosters to be used by multiple tasks in calling applications.
EXGBoost.predict/2andEXGBoost.inplace_predict/2both use dirty CPU-bound NIF schedulers- This prevents blocking the BEAM scheduler but does not provide thread safety
- Concurrent access to the same booster from multiple processes can cause undefined behavior
A Booster can be serialized to a file using EXGBoost.write_* and loaded from a file
using EXGBoost.read_*. The file format can be specified using the :format option
which can be either :json or :ubj. The default is :json. If the file already exists, it will NOT
be overwritten by default. Boosters can either be serialized to a file or to a binary string.
Boosters can be serialized in three different ways: configuration only, configuration and model, or
model only. dump functions will serialize the Booster to a binary string.
Model and weight exports use XGBoost's portable model representation in the
requested JSON or UBJ format. It includes trees and the objective; training
configuration can be saved separately with the configuration API. Readers also
accept legacy serialized snapshots. Configuration files are intended for the
same XGBoost version.
read/write- File.load/dump- Binary buffer.
config- Save the configuration only.weights- Save the model parameters only. Use this when you want to save the model to a format that can be ingested by other XGBoost APIs.model- Save the portable model representation.
EXGBoost.plot_tree/2 is the primary entry point for plotting a tree from a trained model.
It accepts an EXGBoost.Booster struct (which is the output of EXGBoost.train/2).
EXGBoost.plot_tree/2 returns a VegaLite spec that can be rendered in a notebook or saved to a file.
EXGBoost.plot_tree/2 also accepts a keyword list of options that can be used to configure the plotting process.
See EXGBoost.Plotting for more detail on plotting.
You can see available styles by running EXGBoost.Plotting.get_styles() or refer to the EXGBoost.Plotting.Styles
documentation for a gallery of the styles.
Saving plot images with path: requires adding {:vega_lite_convert, "~> 1.0.1"}
to your application. Returning a Vega spec, training, and prediction do not need
that additional native image-conversion runtime.
See the example Notebooks in the left sidebar (under the Pages tab) for more examples and tutorials
on how to use EXGBoost.
EXGBoost 0.6 targets XGBoost 3.4.2, Elixir 1.17+, and OTP 26+. Releases provide CPU archives for these native targets:
| Platform | Architectures | Build baseline |
|---|---|---|
| Linux (glibc) | x86_64, aarch64 | Ubuntu 22.04 / glibc 2.35 |
| macOS | x86_64, aarch64 | macOS 15 Intel / macOS 14 Apple Silicon |
Mix downloads the matching archive and verifies its SHA256 checksum. The NIF,
XGBoost, and dependency license notices are packaged together. macOS archives
include the OpenMP runtime, so consumers do not need Homebrew. Linux consumers
need the standard C++ and OpenMP runtimes (libstdc++6 and libgomp1 on Debian
and Ubuntu). Windows, musl, and CUDA archives are not currently provided.
Unsupported platforms fall back to a source build when the toolchain supports it.
Both macOS archives require macOS 14.0 or newer because of the bundled OpenMP runtime. CI checks the minimum OS recorded in every packaged library.
Install Git, a C11/C++ compiler, Make, and CMake 3.18+. On macOS install the
Xcode command line tools and brew install libomp for source builds.
mix deps.get
EXGBOOST_BUILD=true mix compile
EXGBOOST_BUILD=true mix quality
EXGBOOST_BUILD=true MIX_ENV=test mix coveralls.htmlUse BUILD_JOBS=4 to adjust upstream build parallelism or USE_OPENMP=OFF
to disable OpenMP. Builds retain an architecture-specific upstream cache under
cache/xgboost; they neither move files out of that cache nor patch upstream
source. See RELEASING.md for precompilation, artifact verification,
checksums, and Hex publishing.
- The XGBoost C API uses C function pointers to implement streaming data types. The Python ctypes library is able to pass function pointers to the C API which are then executed by XGBoost. Erlang/Elixir NIFs do not have this capability, and as such, streaming data types are not supported in EXGBoost.
- CUDA support
- Collective API?
Licensed under an Apache-2 license.
Contributor PRs, dependency updates, and native API changes run the CI and four-platform packaging workflows. Both source builds and archived libraries are tested. See CONTRIBUTING.md for NIF development and RELEASING.md for the release procedure.