How to gradually build up a neural network in Matlab?

Question

Suppose I have a very large train set, so Matlab hangs during training or there is not enough memory to store the train set.

Is it possible to break the training kit into parts and train the network in parts?

Is it possible to train a network one pattern at a time (one after another)?

Answer 1

You can simply manually divide the data set into batches and train them one by one:

for bn = 1:num_batches inputs = <get batch bn inputs>; targets = <get batch bn targets>; net = train(net, inputs, targets); end 

Although the batch size should be more than 1, but in any case, this should reduce the memory consumption for training.

In the case of trainlm training, alogrithm can help net.efficiency.memoryReduction optim. Also, instead of the standard trainlm algorithm trainlm you can try less memory-consuming ones, like trainrp . For more information on learning algorithms, check out the Matlab documentation page . I suggested that you use the appropriate Matlab toolkit for neural networks.

As for learning one pattern at a time, you can try a Google search for the stochastic gradient descent algorithm. But it seems that this is not a set of default learning algorithms in the toolbar.