Gradient Descent in Matlab

Question

I take a machine learning class in the course. Machine science is a pretty nice area for me. In the first programming exercise, I have some difficulties in a decent gradient algorithm. If anyone can help me, I will be grateful.

Here are the instructions for updating thetas;

“The gradient view is implemented in the gradientDescent.m gradient view. The loop structure has been written for you, and you only need to update it to θ in each iteration.

    function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
    %GRADIENTDESCENT Performs gradient descent to learn theta
    %   theta = GRADIENTDESENT(X, y, theta, alpha, num_iters) updates theta by 
    %   taking num_iters gradient steps with learning rate alpha

   % Initialize some useful values
   m = length(y); % number of training examples
   J_history = zeros(num_iters, 1);

   for iter = 1:num_iters

% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
%               theta. 
%
% Hint: While debugging, it can be useful to print out the values
%       of the cost function (computeCost) and gradient here.
%
    % ============================================================

% Save the cost J in every iteration    
J_history(iter) = computeCost(X, y, theta);

end

end

So, here is what I did to update thetas at the same time;

    temp0 = theta(1,1) - (alpha/m)*sum((X*theta-y));
    temp1 = theta(2,1) - (alpha/m)*sum((X*theta-y).*X);
    theta(1,1) = temp0;
    theta(2,1) = temp1;

I get an error while running this code. Can anybody help me?

Answer 1

, Error using .* Matrix dimensions must agree. Error in gradientDescent (line 20) temp1 = theta(2,1) - (alpha/m)*sum((X*theta-y).*X);, , .* . , :

size(X*theta-y)
size(X)

(X*theta-y).*X, X*theta-y X . , .

Answer 2

, :

theta = theta - (alpha/m) * (X' * (X * theta - y));

theta = theta - (alpha/m) * ((X * theta - y)' * X)';

.

:

---

:

θ:

, X, y θ:

• m =
• n = + 1

• m = 5 ( )
• n = 4 ( + 1)
• X = m x n matrix
• y = m x 1
• θ = n x 1
• x ⁱ - ^th
• x _j - j ^th

• h(x) = ([X] * [θ]) (m x 1 )
• h(x)-y = ([X] * [θ] - [y]) (m x 1 )

. , m x 1 :

θ _j, (m ), j ^th set X. , E, j ^th . (, , ) θ _j. j . :

:

• [E]' x [X] , E '- 1 x m, X - m x n. , .

, :

:

Answer 3

theta = theta - (alpha/m) * (X' * (X * theta - y));

Answer 4

temp0 = theta(1,1) - (alpha/m)*sum((X*theta-y));
temp1 = theta(2,1) - (alpha/m)*sum((X*theta-y).*X(:,2));
theta(1,1) = temp0;
theta(2,1) = temp1;

Or you can use the code below. It is easier. There are only two parameters: theta1 and theta2. But if there are more options, it is much better.

for i=1:2
    theta(i) = theta(i) - (alpha/m)*sum((X*theta-y).*X(:,i));
end

Answer 5

In this matter it is worth noting:

X = [ones(m, 1), data(:,1)]; 

So

theta = theta - (alpha / m) * (X' * (X * theta - y));

and

temp0 = theta(1, 1) - (alpha / m) * sum((X * theta - y));
temp1 = theta(2, 1) - (alpha / m) * sum((X * theta - y) .* X(:, 2));
theta(1, 1) = temp0;
theta(2, 1) = temp1;

both are correct