MPI-IO: Subarray Recording

Question

I am starting to use MPI-IO and trying to write a very simple example of what I would like to do with it; however, although this is simple code, and I was a little inspired by the examples I read here and there, I get a segmentation error that I don't understand.

The logic of the code part is very simple: each thread processes a local array, which is part of the globlal array that I want to write. To do this, I create a subarray type using MPI_Type_Create_Subarray . Then I just open the file, set the view and try to write the data. I get a segmentation error during MPI_File_Write_All .

Here is the code:

 program test implicit none include "mpif.h" integer :: myrank, nproc, fhandle, ierr integer :: xpos, ypos integer, parameter :: loc_x=10, loc_y=10 integer :: loc_dim integer :: nx=2, ny=2 real(8), dimension(loc_x, loc_y) :: data integer :: written_arr integer, dimension(2) :: wa_size, wa_subsize, wa_start call MPI_Init(ierr) call MPI_Comm_Rank(MPI_COMM_WORLD, myrank, ierr) call MPI_Comm_Size(MPI_COMM_WORLD, nproc, ierr) xpos = mod(myrank, nx) ypos = mod(myrank/nx, ny) data = myrank loc_dim = loc_x*loc_y wa_size = (/ nx*loc_x, ny*loc_y /) wa_subsize = (/ loc_x, loc_y /) wa_start = (/ xpos, ypos /)*wa_subsize call MPI_Type_Create_Subarray(2, wa_size, wa_subsize, wa_start & , MPI_ORDER_FORTRAN, MPI_DOUBLE_PRECISION, written_arr, ierr) call MPI_Type_Commit(written_arr, ierr) call MPI_File_Open(MPI_COMM_WORLD, "file.dat" & & , MPI_MODE_WRONLY + MPI_MODE_CREATE, MPI_INFO_NULL, fhandle, ierr) call MPI_File_Set_View(fhandle, 0, MPI_DOUBLE_PRECISION, written_arr & , "native", MPI_INFO_NULL, ierr) call MPI_File_Write_All(fhandle, data, loc_dim, MPI_DOUBLE_PRECISION & , MPI_INFO_NULL, ierr) call MPI_File_Close(fhandle, ierr) call MPI_Finalize(ierr) end program test 

Any help would be greatly appreciated!

Answer 1

The final argument MPI_FILE_WRITE_ALL before the error output argument is an MPI status object, not an MPI information object. Thus, a call with MPI_INFO_NULL is erroneous. If you are not interested in the status of the write operation, you must pass MPI_STATUS_IGNORE . Making a call using MPI_INFO_NULL may work in some MPI implementations due to the specifics of defining both constants, but then fail in others.

For example, in Open MPI, MPI_INFO_NULL declared as:

 parameter (MPI_INFO_NULL=0) 

When passed instead of MPI_STATUS_IGNORE it calls C MPI_FILE_WRITE_ALL with a status argument that points to a read-only memory location that contains the value MPI_INFO_NULL (the way Fortran implements constant passing by address). When the C function is finished, it tries to populate the state object, which leads to an attempt to write to read-only memory and ultimately leads to a segmentation error.

---

When writing new Fortran programs, it is recommended that you do not use the very old mpif.h interface, since it does not provide error checking. Rather, you should use the mpi module or even mpi_f08 when more MPI implementations become compatible with MPI-3.0. Therefore, the beginning of your program should look like this:

 program test use mpi implicit none ... end program test 

Once you use the mpi module instead of mpif.h , the compiler can check the parameter types for some MPI calls, including MPI_FILE_SET_VIEW , and determine the error:

 test.f90(34): error #6285: There is no matching specific subroutine for this generic subroutine call. [MPI_FILE_SET_VIEW] call MPI_File_Set_View(fhandle, 0, MPI_DOUBLE_PRECISION, written_arr & -------^ compilation aborted for test.f90 (code 1) 

The reason is that the second argument MPI_FILE_SET_VIEW is of type INTEGER(KIND=MPI_OFFSET_KIND) , which is 64-bit on most modern platforms. The constant 0 is just an INTEGER type and therefore is 32-bit on most platforms. It happens that with mpif.h compiler passes a pointer to an INTEGER constant with a value of 0 , but the routine interprets this as a pointer to a larger integer and interprets neighboring values ​​as part of the constant value. So the zero that you pass as the offset inside the file ends with a non-zero value.

Replace 0 in the MPI_FILE_SET_VIEW call with 0_MPI_OFFSET_KIND or declare a constant of type INTEGER(KIND=MPI_OFFSET_KIND) and a value of zero, and then pass it.

 call MPI_File_Set_View(fhandle, 0_MPI_OFFSET_KIND, MPI_DOUBLE_PRECISION, ... 

or

 integer(kind=MPI_OFFSET_KIND), parameter :: zero_off = 0 ... call MPI_File_Set_View(fhandle, zero_off, MPI_DOUBLE_PRECISION, ... 

Both methods result in an output file of 3200 bytes in size (as expected).