I don’t think it really makes sense to talk about “overhead” when switching context to GPU.
On the processor, context switching is performed in software, by means of a function in the kernel called a "scheduler." A scheduler is a regular code, a sequence of machine instructions that a processor must execute, and the time taken to start the scheduler does not waste time on “useful” work.
, , , , , , , " " - . .
. SuperUser.