Cropping in Keras

Question

I am trying to create a neural network using Keras with priority in prediction performance, and I cannot get sufficiently high accuracy by further reducing the number of layers and nodes per layer. I noticed that a very large part of my weights are actually zero (> 95%). Is there a way to crop dense layers in the hope of shortening forecast time?

Answer 1

Not selected path :(

There is currently no easy (dedicated) way to do this with Keras.

The discussion continues at https://groups.google.com/forum/#!topic/keras-users/oEecCWayJrM .

You may also be interested in this document: https://arxiv.org/pdf/1608.04493v1.pdf .

Answer 2

Take a look at Keras Surgeon: https://github.com/BenWhetton/keras-surgeon

I have not tried it myself, but the documentation states that it has functions for deleting or inserting nodes.

In addition, after looking at some cropping work, it seems that many researchers create a new model with fewer channels (or fewer layers), and then copy the weights from the original model to the new model.

Answer 3

If you set the individual weight to zero, will this prevent the update during back distribution? Should this weight not remain zero from one era to another? Therefore, before training, you set the initial weights to non-zero values. If you want to “delete” the entire node, simply set all the weights at the output of this node to zero, and this will prevent any influence of these nodes on the output during training.

Answer 4

See this special tool for Keras. https://www.tensorflow.org/model_optimization/guide/pruning

As you can see from the review, support for delay improvements is under development.