Yes.
Set the text property to "*" and the ErrorMessage property for the error message. Sort of.
<asp:RequiredFieldValidator id="ValidateMyField" runat="server" text="*"
errormessage="Hey, you must really specify something"
controltovalidate="YourControl" />
For display, you can use display="Dynamic"to have * only take up space on the actual error.