Skip to content
forked from NVIDIA/nccl

About

Optimized primitives for collective multi-GPU communication

Resources

Contributing

Stars

13 stars

Watchers

1 watching

Forks

 
 

Repository files navigation

NCCL for Windows

Don't open Issues for general NCCL questions or non Windows related problems. Only Windows specific issues. Any Issue opened that is not Windows specific will be closed automatically.

Building from source:

PC: A Visual Studio 2019 or newer is required to launch the compiler x64 environment. The installation path is referred in the instructions as VISUAL_STUDIO_INSTALL_PATH.

RTX Spark: A Visual Studio 2026 or newer is required to launch the compiler arm64 environment. The installation path is referred in the instructions as VISUAL_STUDIO_INSTALL_PATH.

CUDA path will be found automatically if you have the bin folder in your PATH, or have the CUDA installation path settled on well-known environment vars like CUDA_ROOT, CUDA_HOME or CUDA_PATH.

If none of these are present, make sure to set the environment variable before starting the build: set CUDA_HOME=CUDA_INSTALLATION_PATH

  1. Open a Command Line (cmd.exe)
  2. Clone the NCCL for Windows repository from nccl-windows-rel branch (NOT MAIN): cd C:\ & git clone --single-branch --branch nccl-windows-rel https://github.com/SystemPanic/nccl-windows.git
  3. Execute (in cmd) VISUAL_STUDIO_INSTALL_PATH\VC\Auxiliary\Build\vcvarsall.bat x64 for PC or VISUAL_STUDIO_INSTALL_PATH\VC\Auxiliary\Build\vcvarsall.bat arm64 for RTX Spark.
  4. Change the working directory to the cloned repository path, for example: cd C:\nccl-windows
  5. Build & install:
#(replace 10 with your desired cpu threads to use in parallel to speed up compilation)
set MAX_JOBS=10

#Replace YOUR_GPUS_ARCHS with your multiple gpu's architectures, for example, RTX 5090 is 120, RTX 3090 is 86, RTX Spark is 121. You can mix different archs if you have distinct GPU archs, for example 86;120
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=YOUR_GPUS_ARCHS -DCMAKE_INSTALL_PREFIX=install
cmake --build build --parallel --target install

After the build, bin, lib and include will be under C:\nccl-windows\install


NCCL

Optimized primitives for inter-GPU communication.

Introduction

NCCL (pronounced "Nickel") is a stand-alone library of standard communication routines for GPUs, implementing all-reduce, all-gather, reduce, broadcast, reduce-scatter, as well as any send/receive based communication pattern. It has been optimized to achieve high bandwidth on platforms using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs or TCP/IP sockets. NCCL supports an arbitrary number of GPUs installed in a single node or across multiple nodes, and can be used in either single- or multi-process (e.g., MPI) applications.

For more information on NCCL usage, please refer to the NCCL documentation.

Build

Note: the official and tested builds of NCCL can be downloaded from: https://developer.nvidia.com/nccl. You can skip the following build steps if you choose to use the official builds.

To build the library :

$ cd nccl
$ make -j src.build

If CUDA is not installed in the default /usr/local/cuda path, you can define the CUDA path with :

$ make src.build CUDA_HOME=<path to cuda install>

NCCL will be compiled and installed in build/ unless BUILDDIR is set.

By default, NCCL is compiled for all supported architectures. To accelerate the compilation and reduce the binary size, consider redefining NVCC_GENCODE (defined in makefiles/common.mk) to only include the architecture of the target platform :

$ make -j src.build NVCC_GENCODE="-gencode=arch=compute_90,code=sm_90"

Install

To install NCCL on the system, create a package then install it as root.

Debian/Ubuntu :

$ # Install tools to create debian packages
$ sudo apt install build-essential devscripts debhelper fakeroot
$ # Build NCCL deb package
$ make pkg.debian.build
$ ls build/pkg/deb/

RedHat/CentOS :

$ # Install tools to create rpm packages
$ sudo yum install rpm-build rpmdevtools
$ # Build NCCL rpm package
$ make pkg.redhat.build
$ ls build/pkg/rpm/

OS-agnostic tarball :

$ make pkg.txz.build
$ ls build/pkg/txz/

Python wheel :

$ # Install uv to create the Python wheel (uv manages Python deps in a venv)
$ # See: https://docs.astral.sh/uv/getting-started/installation/
$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ # Build NCCL Python wheel (this also builds the .txz archive as an intermediate)
$ make pkg.python_wheel.build
$ ls build/pkg/python_wheel/

Tests

Tests for NCCL are maintained separately at https://github.com/nvidia/nccl-tests.

$ git clone https://github.com/NVIDIA/nccl-tests.git
$ cd nccl-tests
$ make
$ ./build/all_reduce_perf -b 8 -e 256M -f 2 -g <ngpus>

Copyright

All source code and accompanying documentation is copyright (c) 2015-2020, NVIDIA CORPORATION. All rights reserved.

About

Optimized primitives for collective multi-GPU communication

Resources

Contributing

Stars

13 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages