There are two lines in which you use 0 for indexing, which you cannot do in Matlab:
r(i,0) = (1/2)*r(i-1,0) + (sum1)*h;
and
r(i,j) = r(i,j-1) + (r(i,j-1) - r(i-1,j-1))/((4^j) - 1);
when j == 1 or i == 1.
I suggest you run your loops starting at 2, and replace the indices i and j with (i-1) and (j-1) respectively.
:
for k = 1:2:2^(i)
sum1 = sum1 + f(a + k*h);
end
k = 1:2:2^i;
tmp = f(a + k*h);
sum1 = sum(tmp);
f_of_x
sin(x)./x