-
Notifications
You must be signed in to change notification settings - Fork 2
MPI Wrappers
Vftrace has wrappers to MPI function calls. These wrappers are instrumented and call a vftr_MPI
version of the function. The wrappers for C and Fortran both call the same vftr_MPI
routine to reduce code duplication and enable easier maintenance.
int MPI_Send(const void *buf, int count, MPI_Datatype datatype,
int dest, int tag, MPI_Comm comm) {
return vftr_MPI_Send(buf, count, datatype, dest, tag, comm);
}
In these vftr_MPI
-functions the communication is executed by calling the corresponding PMPI function:
int vftr_MPI_Send(const void *buf, int count, MPI_Datatype datatype,
int dest, int tag, MPI_Comm comm) {
// disable profiling based on the Pcontrol level
if (vftrace_Pcontrol_level == 0) {
return PMPI_Send(buf, count, datatype, dest, tag, comm);
} else {
long long tstart = vftr_get_runtime_usec();
int retVal = PMPI_Send(buf, count, datatype, dest, tag, comm);
long long tend = vftr_get_runtime_usec();
vftr_store_sync_message_info(send, count, datatype, dest, tag, comm, tstart, tend);
return retVal;
}
}
The PMPI_
symbols do the same as their MPI_
counterpart. This way, the MPI functions as used by the application are instrumented. The functionality inside the wrapper enables in-depth MPI sampling. The vftrace MPI-wrapper record which ranks are communicating (who sends, who receives), the message size, message type, and communication time. Non blocking communication is sampled by registering the non blocking call's request and checking for completion from time to time in the background. Persistent requests are registered and handled like normal non blocking calls upon calling MPI_Start
or MPI_Startall
. For collective communications inter- and intra-communicators are distinguished and handled to reflect their unique communication patterns. Special buffers like MPI_IN_PLACE
are taken care of and handled accordingly.
Function | Type | Blocking? | vftr-handling | C-wrapper | C-Regtest | vftr-c2F | F-wrapper | F-Regtest | F08 wrapper | F08-Regtest |
---|---|---|---|---|---|---|---|---|---|---|
94,94% | 94,94% | 94,94% | 94,94% | 94,94% | 94,94% | 35,44% | 30,38% | |||
MPI_Send | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
MPI_Bsend | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
MPI_Ssend | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
MPI_Rsend | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
MPI_Recv | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Mrecv | P2P | Blocking | No | No | No | No | No | No | No | No |
MPI_Isend | P2P | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Ibsend | P2P | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Issend | P2P | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Irsend | P2P | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Irecv | P2P | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Imrecv | P2P | Non-Blocking | No | No | No | No | No | No | No | No |
MPI_Sendrecv | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Sendrecv_replace | P2P | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Probe | Probing | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Iprobe | Probing | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Mprobe | Probing | Blocking | No | No | No | No | No | No | No | No |
MPI_Improbe | Probing | Non-Blocking | No | No | No | No | No | No | No | No |
MPI_Send_init | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Bsend_init | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Ssend_init | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Rsend_init | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Recv_init | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Start | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Startall | Persistent | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Test | Test/Wait | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Testany | Test/Wait | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Testsome | Test/Wait | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Testall | Test/Wait | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Wait | Test/Wait | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Waitany | Test/Wait | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Waitsome | Test/Wait | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Waitall | Test/Wait | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Request_free | Test/Wait | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Barrier | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Bcast | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ibcast | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Gather | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Igather | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Gatherv | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Igatherv | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Scatter | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iscatter | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Scatterv | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iscatterv | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Allgather | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iallgather | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Allgatherv | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iallgatherv | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Alltoall | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ialltoall | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Alltoallv | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ialltoallv | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Alltoallw | Collective | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ialltoallw | Collective | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Allreduce | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iallreduce | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Reduce | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ireduce | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Reduce_scatter_block | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ireduce_scatter_block | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Reduce_scatter | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Ireduce_scatter | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Scan | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iscan | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Exscan | GlobalReduction | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Iexscan | GlobalReduction | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Put | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Rput | Onesided | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Get | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Rget | Onesided | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Accumulate | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Raccumulate | Onesided | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Get_accumulate | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Rget_accumulate | Onesided | Non-Blocking | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
MPI_Fetch_and_op | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Compare_and_swap | Onesided | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Init | Environment | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Finalize | Environment | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
MPI_Pcontrol | Environment | Blocking | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |