You need to consider the context in order to decide if it is proper / safe to use WeakHashMap.
Here's an example where WeakHashMap will not work (pseudocode)
Map<Name, Details> map = ...
do for ever:
name = get name from user
if lookup:
details = map.get(name)
display details
else if create:
details = get details from user
map.add(name, details)
Using WeakHashMap, there is a risk that the records will drop out of the table and the search for users will fail. There is no risk with HashMap.
There is also the problem that WeakReference and everything built on it are more expensive than regular links. They use more space and time. Moreover, the Reference classes incur overhead every time the GC encounters them, which can increase the GC pause time.
However, the run-time overhead problem should usually be secondary to the correctness problem.