How to calculate fan and fan in Xavier initialization for neural networks?

Question

The variations I found in initializing Xavier for weights in a neural network all mention fan-in and fan-out ; Could you tell us how these two parameters are calculated? In particular, for these two examples:

1) initialization of the convolutional layer weights with the shape filter [5, 5, 3, 6] (width, height, input depth, output depth);

2) initialization of the weights of a fully connected layer with the form [400, 120] (ie, mapping 400 input variables to 120 output variables).

Thanks!

Answer 1

I understand that the fan in and out of the convolutional layer is defined as:

fan_in = n_feature_maps_in * receptive_field_height * receptive_field_width fan_out = n_feature_maps_out * receptive_field_height * receptive_field_width / max_pool_area 

where receptive_field_height and receptive_field_width correspond to the characteristics of the conv layer under consideration, and max_pool_area is the product of the height and width of the maximum union following the convolution layer.

Please correct me if I am wrong.

Source: deeplearning.net