Massage order MPSCNN

Question

The Metal Performance Shader mechanism provides support for creating custom convolutional neural networks. When creating, for example, MSPCNNConvolution this requires the 4D weight tensor as the init parameter, which is represented as a 1D swimming pointer.

 init(device: MTLDevice, convolutionDescriptor: MPSCNNConvolutionDescriptor, kernelWeights: UnsafePointer<Float>, biasTerms: UnsafePointer<Float>?, flags: MPSCNNConvolutionFlags) 

The documentation has something to say about the 4D tensor

The filter weight layout is designed so that it can be reinterpreted as a 4D tensor (array) Weight [outputChannels] [kernelHeight] [kernelWidth] [inputChannels / groups]

Unfortunately, this information does not tell me how to organize a 4D array into a one-dimensional Float pointer.

I tried to arrange the scales as required by BNNS , but with no luck.

How to correctly represent a 4D tensor (array) as a 1D Float pointer (array)?

PS: I tried to organize it as an array of C and get a pointer to a flat array, but it did not work.

UPDATE

@RhythmicFistman: This is how I saved it in a simple array that I can convert to UsafePointer<Float> (but does not work):

 var output = Array<Float>(repeating: 0, count: weights.count) for o in 0..<outputChannels { for ky in 0..<kernelHeight { for kx in 0..<kernelWidth { for i in 0..<inputChannels { let offset = ((o * kernelHeight + ky) * kernelWidth + kx) * inputChannels + i output[offset] = ... } } } } 

Answer 1

Ok, so I figured it out. Here are two python functions that I use to convert my convolutions and fully related matrices.

 # shape required for MPSCNN [oC kH kW iC] # tensorflow order is [kH kW iC oC] def convshape(a): a = np.swapaxes(a, 2, 3) a = np.swapaxes(a, 1, 2) a = np.swapaxes(a, 0, 1) return a # fully connected only requires ax/y swap def fullshape(a): a = np.swapaxes(a, 0, 1) return a 

Answer 2

This is what I recently had to do for Caffe weights, so I can provide a Swift implementation for how I reordered them. The following function takes a Caffe array of Floats for convolution (in the [c_o] [c_i] [h] [w] order) and reorders those that Metal expects ([c_o] [h] [w] [c_i] order):

 public func convertCaffeWeightsToMPS(_ weights:[Float], kernelSize:(width:Int, height:Int), inputChannels:Int, outputChannels:Int, groups:Int) -> [Float] { var weightArray:[Float] = Array(repeating:0.0, count:weights.count) var outputIndex = 0 let groupedInputChannels = inputChannels / groups let outputChannelWidth = groupedInputChannels * kernelSize.width * kernelSize.height // MPS ordering: [c_o][h][w][c_i] for outputChannel in 0..<outputChannels { for heightInKernel in 0..<kernelSize.height { for widthInKernel in 0..<kernelSize.width { for inputChannel in 0..<groupedInputChannels { // Caffe ordering: [c_o][c_i][h][w] let calculatedIndex = outputChannel * outputChannelWidth + inputChannel * kernelSize.width * kernelSize.height + heightInKernel * kernelSize.width + widthInKernel weightArray[outputIndex] = weights[calculatedIndex] outputIndex += 1 } } } } return weightArray } 

Based on the visualization of my layer, this seems to generate the correct convolution results (corresponding to those produced by Caffe). I believe that it also takes grouping into account correctly, but I need to verify this.

Tensorflow has a different order than Caffe, but you should be able to change the math in the inside of the loop to take this into account.

Answer 3

The documentation here assumes some experience with C. In this context, [x] [y] [z] is usually collapsed into the 1st array when x, y and z are constants known at compile time. When this happens, the z component changes most rapidly, followed by y, followed by x - out.

If we have [2] [2] [2], it collapses to 1D as:

 { a[0][0][0], a[0][0][1], a[0][1][0], a[0][1][1], a[1][0][0], a[1][0][1], a[1][1][0], a[1][1][1] }