What algorithms were proposed for studying the architecture of a deep neural network?

Question

Yoshua Benhgio A study of deep architectures for AI in the book mentions that

we must [...] strive to develop learning algorithms that use data to determine the depth of the final architecture.

Does anyone know of any algorithms proposed so far to achieve this?

This question does not concern successful algorithms - in fact, it seems that at the moment they are not. The goal of this question is to combine every algorithm that has ever been proposed so that everyone who is interested in the topic does not need to spend months searching for them.

So far I have come across:

---

• tiling algorithm
  • advantage: adds layers as well as units
  • Caution: only for learning Boolean functions that are not too relevant for applied tasks.

---

• Genetic algorithms (kindly provided by vzn user ):
  • Search for an optimal neural network architecture using genetic algorithms
  • The use of genetic algorithms to select the artificial neural network architecture with forcing advantage
  • : teaches continuous functions, i.e. relates to applied tasks.
  • caveat: computationally very expensive

Answer 1

, Cascade Correlation, .

, C - , , .

Answer 2

, NEAT EANT "" . , ( ).

, , - , . .

, , , , . "" , .

Answer 3

( ) ( TSAGANN).

(, ).

:

• ( ) ANN ANF, ANF, bacpropagation;

• (SA) (, ANN).

, SA, ANN , .

( ) ANN GA. backpropagation, GA, ANN. . GA . ( , , ).

PROBEN1, , , .

" " ( ) ANN.