Just use
$ rename 's/^#//' *
use -n to verify that what you think will happen will actually happen. In your example, you will find out about the wrong quotes used (reverse) in the error message
-bash: s/#//g: No such file or directory
bash is trying to execute a command with a name s/#//g.
, g () , #, , .