In my opinion, your colleague was mistaken when he identified himself as you, no matter what the actions were - he should have been formed as his own named user for the sake of a verification trace.
At the same time - if this user had the correct privileges to push the code to the repo (as his own named user), and there were no hook settings to verify that the incoming commit matches the person performing the push - then I would say no , there is no way to distinguish who committed these commits.
Here is a preliminary trick that will verify that the person pressing the code is also the author specified in the commit.