Python novice understanding some code

Question

Here is a Python view of a neural neural network I'm trying to understand

class Network(object): def __init__(self, sizes): self.num_layers = len(sizes) self.sizes = sizes self.biases = [np.random.randn(y, 1) for y in sizes[1:]] self.weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])] 

Here is my understanding:

• self.num_layers = len(sizes) : return the number of elements in sizes
• self.sizes = sizes : assign sizes of own instances to size parameters
• self.biases = sizes : generate an array of elements from the standard normal distribution (denoted by np.random.randn(y, 1) )

What are the following linear calculations?

 self.weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])] 

I am new to Python. Can this code be used in a Python shell so that I can better understand calling each line separately?

Answer 1

The zip() function combines elements from each iterable; zip('foo', 'bar') , for example, will produce [('f', 'b'), ('o', 'a'), ('o', 'r')] ; each element in two rows was combined into three new tuples.

zip(sizes[:-1], sizes[1:]) then creates pairs of elements in the sizes sequence with the next element, because you combine all the elements except the last one ( sizes[:-1] ) with all elements except the first ( sizes[1:] ). This connects the first and second elements together, then the second and third, etc. Up to the last two elements.

For each such pair, an arbitrary sample is created using a list comprehension . Thus, for each pair x, y , a new two-dimensional matrix is ​​created with random values ​​divided into rows y and columns x .

Note that the biases value uses only sizes[1:] , except the first, to create y -by-1 matrices for each such size.

A quick demonstration of these concepts:

 >>> zip('foo', 'bar') [('f', 'b'), ('o', 'a'), ('o', 'r')] >>> zip('foo', 'bar', 'baz') # you can add more sequences [('f', 'b', 'b'), ('o', 'a', 'a'), ('o', 'r', 'z')] >>> sizes = [5, 12, 18, 23, 42] >>> zip(sizes[:-1], sizes[1:]) # a sliding window of pairs [(5, 12), (12, 18), (18, 23), (23, 42)] # 0, 1 .. 1, 2 .. 2, 3 .. 3, 4 element indices into sizes >>> 

Answer 2

self.weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])] will call the randn function with parameters x, y that are the results of the zip(sizes[:-1], sizes[1:]) operation zip(sizes[:-1], sizes[1:])

If we look at the list l=[1, 2, 3, 4] l[:-1] will return [1, 2, 3] and l[1] will give [2, 3, 4] The zip operation on l[:-1], l[1] will make pairs [(1, 2), (2, 3), (3, 4)] . Then the pairs will be passed to the randn function

Of course, you can always enter code in the python shell, this will give you a better understanding;)

Answer 3

This is what is called list comprehension . You can create the same effect if you use the usual for loop:

 self.weights = [] for x, y in zip(sizes[:-1], sizes[1:]): self.weights.append(np.random.randn(y, x)) 

Now with this loop, you can see that self.weights is actually just a bunch of np.random.randn(y, x) , where y and x defined for each x and y in zip(sizes[:-1], sizes[1:]) . You can just say it to yourself when you read the list comprehension: self.weights = [np.random.randn(y, x)) for x, y in zip(sizes[:-1], sizes[1:])] . The word order finally makes sense. If you did not know, zip is a function that returns a list of tuples of each corresponding element in its arguments. For example, zip([1, 2, 3, 4], [4, 3, 2, 1]) will return [(1, 4), (2, 3), (3, 2), (4, 1)] . (In Python3, this is actually a tuple generator)

Answer 4

This actually creates two random variables x and y. one for connections from the 1st layer to the second layer of neurons. and others for the second layer of neurons to output the layer of neurons. dimensions (-1) mean all connections in the vector, except the last one, which is the 1st layer for hidden layers. and dimensions (1) are all connections except the 1st element in the vector. This is the weight of the connections from hidden to the output level.

Note. Joins or tuples are formed using the ZIP function.

Answer 5

If you know C ++, here is the conversion for self.weights = [np.random.randn(y, x) for x, y in zip(sizes[:-1], sizes[1:])] to C + + which I did. It uses the Eigen C ++ library instead of the Numpy Python library. You call it by typing Weights(weights, sizes); in main (). The parameters for the Weights function consist of going through the reference list of matrices (weights) and Vector (sizes). Passing by reference, indicated by the "&" symbol, basically means that the value of the weights will change both in the function and in the main loop. This is different from the pass by value because pass by value only changes the value of the weights in the function. If you are trying to fully reproduce this, you will need to enter #include <list> , #include<Eigen/Dense> , using namespace std; and using namespace Eigen; .

 void Weights(list<MatrixXd> &weights, VectorXi sizes){ int x,y; for(int i=0; i < sizes.rows()-1;i++){ y=sizes[i+1]; //sizes[1:] x=sizes[i]; //sizes[:-1] weights.push_back(MatrixXd::Random(y,x)); //np.random.randn(y,x) } }