What does the "relu" mean in tf.nn.relu?

Question

The API documentation says: "Computes a straightened linear value."

Is Re (ctified) L (inear) ... what is U then?

Answer 1

Re (ctified) L (inear) (U) nit

Usually a layer in a neural network has some input, say a vector, and multiplies it by a weight matrix, resulting in again in a vector.

( , float) . , , , , , , . .

, 0, , , 0 . ( " " ) relu.

Answer 2

Friesel Relu.

• .

Relu graph: , .

f(x) = max(0,x) .

2. ReLU ! , x(1-x).

   1, x > 0

   0

, . , backpropagation!