If you use it in only one place and do not plan to provide internationalization, leave it built-in.
If you use it in more than one place, especially if the message becomes part of your interface (so you can check the message when you catch the exception, although you really have to use a separate error code or subclasses for this), make it permanent (if necessary public )
If you need to localize the message, enter it in the message resource file.
Thilo source share