When performing CNN tests, I found that most of the time is spent on fully connected layers. But when it comes to computing computational complexity, I found that:
O(conv) = N*(D * (W+P) * (H+P) * h *w)/S
O(fully_connected) = D*W*H*N
Where
D = Dimensions Input
W,w = Width Input, width Filter
H, h = Height Input, height Filter
S = Stride
P = Padding
N = number of outputs
As an example, I have 1024x11x11 feature map input DxWxH, a 5x5 filter h,wwithout filling p, and with Stride S of 1andnumber of outputs N shall be 512
This leads to the following calculation for convolution:
O (conv) = 512 * (1024 * 11 * 11 * 5 * 5) / 1 = 1 585 971 200
If the same input is used for a fully connected level, and the desired output is 512, then:
O (fully connected) = 512 * 1024 * 11 * 11 = 63,438,848
, conv- , ? ?