In my project, one of my goals is to find outliers in the data of aircraft engines and decided to use the Replicator neural network for this and read the next report about it ( http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.12 .3366 & rep = rep1 & type = pdf ), and I had a small problem with the step-by-step function (p. 4, Fig. 3) and the prediction values.
The explanation of the replicator neural network is best described in the above report, but as the background that the replicator neural network I built has the same number of outputs as the inputs and has 3 hidden layers with the following activation functions:
Hidden layer 1 = tanh sigmoid S1 (θ) = tanh, Hidden layer 2 = stepwise, S2 (θ) = 1/2 + 1 / (2 (k - 1)) {summation of each variable j} tanh [a3 (θ - j / N)] Hidden layer 3 = tanh sigmoid S1 (θ) = tanh, Output layer 4 = normal sigmoid S3 (θ) = 1/1 + e ^ -θ I implemented the algorithm and seems to be training (since the average square error steadily decreases during training). The only thing I don’t understand is how the predictions are made when the average level is applied using a stepwise activation function, since it activates the activation of 3 average nodes with specific discrete values (for example, my last activation on 3 average was 1.0 , -1.0, 2.0), this leads to the fact that these values will be transmitted forward, and I get very similar or exactly the same forecasts every time.
The section in the report on page 3-4 describes the algorithm best, but I have no idea what I need to do to fix this, I also have little time :(
Any help would be greatly appreciated.
thank
source
share