Add documentation for communication modes on send operations

docs/api/core.rst

@@ -2,23 +2,31 @@ Core
 ====
 
 .. list-table:: MPI API Support
-    :widths: 40 30 15 15
+    :widths: 20 30 15
     :header-rows: 1
 
     * - MPI
       - ``KokkosComm::``
       - ``Kokkos::View``
-    * - MPI_Send
-      - send
+    * - ``MPI_Send``
+      - ``send`` or ``send<CommMode::Standard>``
       - ✓
+    * - ``MPI_Rsend``
+      - ``send<CommMode::Ready>``
       - ✓
     * - MPI_Recv
       - recv
+    * - ``MPI_Ssend``
+      - ``send<CommMode::Synchronous>``
       - ✓
+    * - ``MPI_Isend``
+      - ``isend`` or ``isend<CommMode::Standard>``
       - ✓
-    * - MPI_Isend
-      - isend
+    * - ``MPI_Irsend``
+      - ``isend<CommMode::Ready>``
       - ✓
+    * - ``MPI_Issend``
+      - ``isend<CommMode::Synchronous>``
       - ✓
     * - MPI_Reduce
       - reduce
@@ -28,10 +36,10 @@ Core
 Point-to-point
 --------------
 
-.. cpp:function:: template <KokkosExecutionSpace ExecSpace, KokkosView SendView> \
+.. cpp:function:: template <KokkosComm::CommMode SendMode = CommMode::Standard, KokkosExecutionSpace ExecSpace, KokkosView SendView> \
                   Req KokkosComm::isend(const ExecSpace &space, const SendView &sv, int dest, int tag, MPI_Comm comm)
 
-    MPI_Isend wrapper
+    Wrapper for ``MPI_Isend``, ``MPI_Irsend`` and ``MPI_Issend``.
 
     :param space: The execution space to operate in
     :param sv: The data to send
@@ -41,11 +49,12 @@ Point-to-point
     :tparam SendView: A Kokkos::View to send
     :tparam ExecSpace: A Kokkos execution space to operate in
     :returns: A KokkosComm::Req representing the asynchronous communication and any lifetime-extended views.
+    :tparam SendMode: A communication mode to use. If unspecified, defaults to a standard ``MPI_Isend``.
 
-.. cpp:function:: template <KokkosExecutionSpace ExecSpace, KokkosView SendView> \
+.. cpp:function:: template <KokkosComm::CommMode SendMode = CommMode::Standard, KokkosExecutionSpace ExecSpace, KokkosView SendView> \
                   void KokkosComm::send(const ExecSpace &space, const SendView &sv, int dest, int tag, MPI_Comm comm)
 
-    MPI_Send wrapper
+    Wrapper for ``MPI_Send``, ``MPI_Rsend`` and ``MPI_Ssend``.
 
     :param space: The execution space to operate in
     :param sv: The data to send
@@ -54,6 +63,7 @@ Point-to-point
     :param comm: the MPI communicator
     :tparam SendView: A Kokkos::View to send
     :tparam ExecSpace: A Kokkos execution space to operate in
+    :tparam SendMode: A communication mode to use. If unspecified, defaults to a standard ``MPI_Send``.
 
 .. cpp:function:: template <KokkosExecutionSpace ExecSpace, KokkosView RecvView> \
                   void KokkosComm::recv(const ExecSpace &space, RecvView &rv, int src, int tag, MPI_Comm comm)
@@ -90,6 +100,22 @@ Collective
 Related Types
 -------------
 
+.. cpp:enum-class:: KokkosComm::CommMode : uint8_t
+
+    A scoped enum to specify the mode of an operation. Buffered mode is not supported.
+
+    .. cpp:enumerator:: KokkosComm::CommMode::Standard
+
+      Standard mode: the MPI implementation decides whether outgoing messages will be buffered. Send operations can be started whether or not a matching receive has been started. They may complete before a matching receive is started. Standard mode is non-local: successful completion of the send operation may depend on the occurrence of a matching receive.
+
+    .. cpp:enumerator:: KokkosComm::CommMode::Ready
+
+      Ready mode: Send operations may be started only if the matching receive is already started.
+
+    .. cpp:enumerator:: KokkosComm::CommMode::Synchronous
+
+      Synchronous mode: Send operations complete successfully only if a matching receive is started, and the receive operation has started to receive themessage sent.
+
 .. cpp:class:: KokkosComm::Req
 
     A wrapper around an MPI_Request that can also extend the lifetime of Views.