Skip to content

Commit

Permalink
Set read/write tracking flags when passing debug array to MPI.Buffer()
Browse files Browse the repository at this point in the history
If a shared-memory array (or part of one) is passed to `MPI.Buffer()` it
will be communicated. This probably implies it is read on at least one
of the distributed-MPI processes, and read on others. To check for
errors as conservatively as possible (I think, but not sure if this is
really true), set the `is_read`, `is_written` and `is_initialized` and
`accessed` flags of an `MPIDebugSharedArray` when it is passed to
`MPI.Buffer()`, so that a shared-memory synchronization is required
before the array is used. This should be a sensible thing to do because:
distributed-MPI communication is done by the root process of each
shared-memory block, so any array that is touched by distributed-memory
communication requires a synchronization before processes in the
shared-memory block (apart from the root process) are allowed to use it.
  • Loading branch information
johnomotani committed Mar 31, 2024
1 parent 61bf925 commit 0cd6007
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions moment_kinetics/src/communication.jl
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,10 @@ end

import MPI: Buffer
function Buffer(A::DebugMPISharedArray)
A.is_initialized .= 1
A.is_read .= true
A.is_written .= true
A.accessed[] = true
return Buffer(A.data)
end

Expand Down

0 comments on commit 0cd6007

Please sign in to comment.