I think your solution is pretty good. My immediate approach would be very similar:
mp=: +/ .* NB. matrix multiplication
XtY=: mp~ |: NB. sum of cross products (monadic is XtX)
proj_mat=: mp %.@XtY mp |:
%.
, . , y %. X
X'X^(-1)X'y
. , %.
, - X.
I=: =@i.@
proj_mat=: mp I %. ] NB. projection matrix
, , LAPACK.