Skip to content

Commit d8121f0

Browse files
authored
in-place conditions for ccl (#552)
1 parent 4ee6e71 commit d8121f0

1 file changed

Lines changed: 22 additions & 6 deletions

File tree

source/elements/oneCCL/source/spec/collective_operations.rst

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,13 @@ If the arguments provided to a communication operation call do not comply to the
4444
Allgatherv
4545
**********
4646

47-
Allgatherv is a collective communication operation that collects data from all the ranks within a communicator into a single buffer. Different ranks may contribute segments of different sizes. The resulting data in the output buffer must be the same for each rank.
47+
Allgatherv is a collective communication operation that collects data from all
48+
the ranks within a communicator into a single buffer. Different ranks may
49+
contribute segments of different sizes. The resulting data in the output buffer
50+
must be the same for each rank.
51+
52+
Allgatherv is in place when ``send_buf == recv_buf + rank_offset``, where
53+
``rank_offset = sum (recv_counts[i])``, for all ``i < rank``.
4854

4955
.. code:: cpp
5056
@@ -100,7 +106,11 @@ return ``event``
100106
Allreduce
101107
*********
102108

103-
Allreduce is a collective communication operation that performs the global reduction operation on values from all ranks of communicator and distributes the result back to all ranks.
109+
Allreduce is a collective communication operation that performs the global
110+
reduction operation on values from all ranks of communicator and distributes
111+
the result back to all ranks.
112+
113+
Allreduce is in-place when ``send_buf == recv_buf``.
104114

105115
.. code:: cpp
106116
@@ -291,8 +301,11 @@ return ``event``
291301
Reduce
292302
******
293303

294-
Reduce is a collective communication operation that performs the global reduction operation
295-
on values from all ranks of the communicator and returns the result to the root rank.
304+
Reduce is a collective communication operation that performs the global
305+
reduction operation on values from all ranks of the communicator and returns
306+
the result to the root rank.
307+
308+
Reduce is in-place when ``send_buf == recv_buf``.
296309

297310
.. code:: cpp
298311
@@ -350,8 +363,11 @@ return ``event``
350363
ReduceScatter
351364
**************
352365

353-
Reduce-scatter is a collective communication operation that performs the global reduction operation
354-
on values from all ranks of the communicator and scatters the result in blocks back to all ranks.
366+
Reduce-scatter is a collective communication operation that performs the global
367+
reduction operation on values from all ranks of the communicator and scatters
368+
the result in blocks back to all ranks.
369+
370+
ReduceScatter is in-place when ``recv_buf == send_buf + rank * recv_count``
355371

356372
.. code:: cpp
357373

0 commit comments

Comments
 (0)