Linear and nonlinear neural network?

Question

I am new to machine learning and neural networks. I know how to build a non-linear classification model, but my current problem has a continuous output. I was looking for information about regression of a neural network, but all I came across was linear regression information - nothing about non-linear cases. Which is strange, because why can someone use neural networks to solve a simple linear regression? Isn't that like killing a nuclear bomb fly?

So my question is: what makes a neural network non-linear? (Hidden layers? Non-linear activation function?) Or am I completely misunderstood the word "linear" - can NN linear regression accurately model datasets that are more complex than y = aX + b? Is the word linear used as the opposite of logistics?

(I plan to use TensorFlow, but the TignorFlow Linear Model Tutorial uses the binary classification problem as an example, so that doesn't help me either.)

Answer 1

, ( ). - http://neuralnetworksanddeeplearning.com/chap4.html.

, .

, . , , . , .

Answer 2

, , . , . , , Feedforward , - , . , :

, , . , , - , . , z, y . , , , y .

, , , , . , , , RProp , (, , ) , .

Answer 3

. , , .

, , , . , .

, - , ReLu, . , .

Answer 4

, - ANN , .

ANN ? .

ANN , ( ), .

, (, ) , , .

ANN? , .

, ( - ) . (, ReLu, ), , .

ANN veeeeeery, , "".

, :)