Deeplearning4j: Iterations, Ages, and ScoreIterationListener

Question

Good day to all.

I am new to Deepleaning4j library, and there are a few things that are still not clear. The concept of "era" is not new, so it is clear that it is a complete learning cycle. My first doubt is related to the concept of "iteration". What is iteration over the training complex? Does this correspond to an analysis of the number of mini-batches of training copies or something else?

In my code, I installed ".iterations (1)"; however, when I run my code, I see a lot:

... ScoreIterationListener - metric on iteration XX - yy.yyyyyy "

So, if I set ".iterations (1)", why do I keep seeing XX values ​​greater than 1? Are there perhaps some differences between the idea of ​​“iteration” as a network configuration parameter and what does “iteration” mean for the ScoreIterationListener class?

Thanks to everyone for any answer or link to useful information.

Best, Mauro.

Answer 1

The DeepLearning4J documentation has a good idea, especially regarding the difference between epoch and iteration.

According to DL4J documentation:

“ iteration is just one update of the parameters of neural networks. Not to be confused with an era that is one complete passage through a dataset. Many iterations can occur before the end of an era. Epoch and iteration are only synonymous if you update your parameters once for each going through the whole data set; if you update mini-packages, they mean different things.Let's say your data has 2 mini-bars: A and B. .numIterations(3) does training, for example AAABBB , and 3 eras look like ABABAB . "

As for your question, and as indicated in this passage, if you installed .iterations(1) and had only one batch, then the iteration would be synonymous with 1 era or one pass through a complete dataset. However, if you update mini-parties, the era and iteration will be slightly different - the iteration will lead to AAABBB, and not to the era that will lead to ABABAB (link to the example above).

Hope this answer and related documentation answers your question!

PS I apologize for the late reply; I came across this question recently!