PyBrain neuron manipulation

Question

Is there a good way to add / remove a neuron and its associated connections to / from a fully connected PyBrain network? Let's say I start with:

from pybrain.tools.shortcuts import buildNetwork net = buildNetwork(2,3,1) 

How can I do this (2,4,1) or (2,2,1) WHILE network, supporting all old scales (and initializing any new ones to be random, how is this done during network initialization)? The reason I want to do this is because I'm trying to use an evolutionary learning strategy to determine the best architecture, and the “mutation” step involves adding / removing nodes with some probability. (Input and output modules must always remain unchanged.)

edit: I found NeuronDecomposableNetwork, which should make it easier, but it seems to me that I need to track neurons and connections separately.

Answer 1

I assume you are running NEAT algorithms? There are two different answers to your question:

• Evolution of open-end network topology: in this case, I recommend encapsulating each neuron in its own “layer” / module and adding / removing them and their network connections iteratively, a bit like this tutorial , except that there will be many more ( single-neural) layers. Remember to call the sortModules() method after each topological change.

• Finding the best topology in a predetermined structure (say, a maximum of 1000 neurons). In this case, it is easier and more efficient to build a complete network at the beginning and just a mask of some connections (for example, using MaskedParameters ). Among others, memetic algorithms (used like this) are designed to search for such topology spaces.

An alternative, as you say, manually manages all the weights (by tracking what is there or using NeuronDecomposableNetwork ), but I do not recommend it.

---

General note: for more complex applications such as yours, as yours rely on, the `buildNetwork 'shortcut is really too limited and you will want to use the Network / Module / Connection API directly.