The grouping of the output signals of the neural network 0.5?

Question

I tried to write a neural network system, but even skipping simple problems like AND / OR / NOR, the results seem to group around 0.5 (for offset -1) and 0.7 (for offset 1). This does not look “wrong” ... 1.1 in the AND pattern seems to be higher than the rest, and 0.0 in the OR looks lower, but they are still grouped, so they are debatable.

I was wondering: a) if there is some obvious mistake that I made, or b) if there are any tips on debugging neural networks ... seeing that you cannot always track where the answer came from ...

Thanks!

Mike

Answer 1

I answered a similar question, and I posted some values ​​that you can check against:

Artificial Neural Network Test

Use these values ​​to test your neural network. Your outputs should match the VERY close values ​​even if you use different languages ​​... you may be able to match at least 6 digits after the decimal without problems.

Answer 2

for a) it is impossible to tell without additional information.

for b) the best way I found is to have a pre-computed set of vectors and weights for a small test version of my network architecture and have some debugging code that uploads the network state to a csv file, so I can look at it in Excel.

The prospective (and reverse) passes of your network must be completely manually calculated for one template and the known initial state of the network, so it’s nice to have a candidate template or a pair, if you can do it manually, are completely listed for your tests. This is not as bad as it seems.

This has the added benefit of reinforcing your understanding of how the algorithm works and making you check your implementation again from a different perspective.

good luck.

PS shameless plugin on backprop blog entry here ...

Answer 3

Thanks to everyone on this page :) I tried everything on this page and it all seemed beautiful.

In the end, Simon’s page was very informative, but Lyric helped the most :) His link led me to the link http://www.generation5.org/content/2001/xornet.asp , I checked all my values ​​to the end and found that they ... stumble ... magically, the next time I ran the program, it magically worked :) (admitted that it was a little less efficient than I would like, taking a lot more Epochs than I did hoped, but it works, and this is important :))

So thanks to everyone who answered :)