How to choose a device when running the CUDA executable?

Question

I connect to the GPU cluster from the outside, and I have no idea how to choose a device to run my CUDA programs.

I know that there are two Tesla GPUs in the cluster, and I would like to choose one of them.

Any ideas how? How do you choose the device you want to use when many computers are connected to the computer?

Answer 1

The canonical way to select a device in the runtime API uses cudaSetDevice . This allows you to configure the runtime to set the lazy context on the nominated device. Prior to CUDA 4.0, this call did not actually create a context; it simply talked about the runtime that the GPU was trying to use. Starting with CUDA 4.0, this call will set the context to the designated GPU during the call. There is also a cudaChooseDevice that will select among the available devices to find one that matches the criteria provided by the caller.

You can list the available GPUs in the system using cudaGetDeviceCount and get their data using cudaGetDeviceProperties . The deviceQuery SDK example shows complete information on how to do this.

However, you may need to be careful about how you choose a GPU on a system with multiple GPUs, depending on the host and driver configuration. In both Linux and Windows, the TCC driver has an option for the column “mark“ exculsive ”, which means that the driver limits each GPU to one active context at a time or calculates a ban, which means that the CUDA program cannot establish a context on this If your code tries to establish a context on a device with forbidden computing or on an exclusive computing device that is used, the result will be an incorrect device error.In a multiprocessor GPU system where the policy is to use an exception Noah exclusivity, the right approach is not to try to select a specific graphics processor, but just to lazy creation of context happened implicitly. The driver automatically selects a free graphics processor to run your code. Status mode of calculating any device can be checked by reading the field cudaDeviceProp.computeMode , by calling cudaGetDeviceProperties . Please note that you can check inaccessible or prohibited GPUs and request their properties, but any operas tion, which requires a context installation fails.

See the execution API documentation in all of these calls.

Answer 2

You can set the CUDA_VISIBLE_DEVICES environment CUDA_VISIBLE_DEVICES to a comma-separated list of device identifiers to display only those devices in the application. Use this either to mask devices or to change the device visibility order so that CUDA runtime lists them in a specific order.