Second MPI_Send hangs if buffer size exceeds 256

Question

int n, j, i, i2, i3, rank, size, rowChunk, **cells, **cellChunk; MPI_Status status; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &size); if(!rank){ printf("\nEnter board size:\n"); fflush(stdout); scanf("%d", &n); printf("\nEnter the total iterations to play:\n"); fflush(stdout); scanf("%d", &j); srand(3); rowChunk = n/size; //how many rows each process will get for(i=1; i<size; i++){ MPI_Send(&n,1, MPI_INT, i, 0, MPI_COMM_WORLD); MPI_Send(&j,1, MPI_INT, i, 7, MPI_COMM_WORLD); } cells = (int**) malloc(n*sizeof(int*)); //create main 2D array for(i=0; i<n; i++){ cells[i] = (int*) malloc(n*sizeof(int)); } for(i=0; i<n; i++){ for(i2=0; i2<n; i2++){ //fill array with random data cells[i][i2] = rand() % 2; } } for(i=1; i<size; i++){ //send blocks of rows to each process for(i2=0; i2<rowChunk; i2++){ //this works for all n MPI_Send(cells[i2+(rowChunk*i)], n, MPI_INT, i, i2, MPI_COMM_WORLD); } } cellChunk = (int**) malloc(rowChunk*sizeof(int*)); for(i=0; i<rowChunk; i++){ //declare 2D array for process zero array chunk cellChunk[i] = (int*) malloc(n*sizeof(int)); } for(i=0; i<rowChunk; i++){ //give process zero it proper chunk of the array for(i2=0; i2<n; i2++){ cellChunk[i][i2] = cells[i][i2]; } } for(i3=1; i3<=j; i3++){ MPI_Send(cellChunk[0], n, MPI_INT, size-1,1,MPI_COMM_WORLD); //Hangs here if n >256 MPI_Send(cellChunk[rowChunk-1], n, MPI_INT, 1,2,MPI_COMM_WORLD); //also hangs if n > 256 ... //Leaving out code that works 

This code works fine if n (array size) is less than or equal to 256. Any more and it hangs on the first MPI_Send. In addition, when sending arrays of array strings to other processes (first MPI_Send), other processes get their data perfectly, even if n> 256. What can lead to the fact that this MPI_Send will hang if the buffer size exceeds 256?

Answer 1

You never receive any messages, so the code will fill in the local space of the MPI buffer, and then a dead end waiting for MPI_Recv (or similar) to run. You will need to insert receive operations so that your messages are actually sent and processed by the recipients.

Answer 2

MPI_Send is a blocking call. The standard provides that MPI_Send can return control already in the message buffer, which can be safely changed. In addition, MPI_Send can wait until some time after the start or end of MPI_Recv.

The MPI implementation that you use most likely performs an impatient message progression if the message is <256 count (with the MPI_INT data type, this will be a 1k message). The message is copied to another buffer, and control returns "earlier". For large (r) messages, the MPI_Send call is not returned until (at least) the corresponding MPI_Recv request is made.

If you publish a full player, you are likely to get a better answer.

Answer 3

MPI_Send "can be blocked until a message is received.", Therefore, most likely, a match will not be reached. You must ensure that MPI_Recv placed in the correct order. Since you did not send your part of the reception, it is impossible to provide details.

You can rebuild your application to make sure that the relevant data is received in order. It may also be convenient for you to use combined MPI_Sendrecv or non-blocking MPI_Isend , MPI_Irecv and MPI_Wait .