CUDA Sync Kernels

Question

Hi, I have doubts about programming in CUDA. I have the following code:

int main () { for (;;) { kernel_1 (x1, x2, ....); kernel_2 (x1, x2 ...); kernel_3_Reduction (x1); // code manipulation host_x1 // Copy the pointer device to host cpy (host_x1, x1, DeviceToHost) cpu_code_x1_manipulation; kernel_ (x1, x2, ....); } } 

So, when are the copies made and how can I make sure that kernel_1, kernel_2 kernel_3 have completed their tasks?

Answer 1

All operations running on the same thread are synchronized. In the above code, all kernels will start one after another. You will need to explicitly specify the threads if you need kernel_1 and kernel_2 to run in parallel.

Answer 2

Use cudaDeviceSynchronize(); where you want all cores to be executed. After this command, you can assume that all kernels and all pending device function calls are executed.