Using Silverlight 5 GPUs for General Mathematics

Question

I am working on a built-in Silverlight browser application that has some fairly intensive calculations, for example, triggering the inverse discrete cosine transform or the fast Fourier transform hundreds of times per second. It would be useful to be able to offload as much as possible onto a computer graphics processor. I know this was a discussion with Silverlight 3 and 4 using pixel shaders, but the consensus was that Silverlight 3/4 did not use hardware acceleration for their pixel shaders, and since their pixel shader language was limited to level 2, this is not would result in most of the increase in productivity, if any.

However, presumably, Silverlight 5 has a much wider range of hardware-accelerated graphics, including a fairly full 3D pipeline. However, I have not heard yet if anyone could use this pipeline to speed up general-purpose mathematical operations (such as FFT, DCT, IDCT, etc.). Has anyone else tried this? Any pointers to where to start looking?

Answer 1

I thought I would send back what I have discovered so far. Short answer: no, I don’t think that the Silverlight 5 3D pipeline can be used for this kind of thing. On the one hand, from what I can say, the pixel shaders and vertex shaders that are part of the pipeline actually execute on the GPU (unlike the 2D shaders in Silverlight 4 that were executed on the processor),

But that said:

(1) Everything I read suggests that receiving data on the GPU is very fast, but for most computers, receiving this data from the GPU is much slower - on the order of milliseconds. It is unlikely that we could, say, load the GPU with the data needed to execute the FFT, execute the FFT, and then pull out the data faster than we could just do it on the CPU.

(2) Silverlight 5 has a very limited set of instructions that it can execute on the GPU. In particular, it is limited to HLSL Level 2 , which has a limited number of instructions and registers. I doubt that it would be possible - at best it would be very difficult and very slow - to simulate an FFT or DCT in these limited instructions.

(3) But even if we could get around these two limitations, from what I can say, Silverlight is not able to read the results of calculations performed by the GPU. Normal XNA (the structure on which Silverlight 3D functions are based) has various GetData () or GetTexture () methods, which I think you can use to read the results of a set of calculations. But these equivalent methods are missing in their versions of Silverlight 5. Of all that I can say in Silverlight 5, the GPU is a recording device. You upload your shaders to it, upload your data, you pull the trigger, and you mache goodbye. Your code will never see these bytes again.

If it turns out that I'm wrong, I will return here and update the answer. But at least for now, it seems like it's a dead end.

[Edit 10/10/11 - According to Shawn Hargreaves from MS, this is not supported in Silverlight 5. His hunch about why (a) it would be difficult to get it to work consistently in all GPU drivers and (b) for all but a tiny class problems with the demo style, it makes no sense. Oh, OK.]