Thread safety is a complex issue. If you want to make the stream of objects safe, do it consciously and write down this choice. People who use your class will be grateful if it is thread safe, when it will simplify their use, but they will curse you if an object that was once thread safe becomes different in a future version. Thread safety, although very good, is not just for Christmas!
So now for your question:
ConcurrentHashMap (at least in Sun current implementation ) works by dividing the base map into several separate buckets. Getting an element does not require any locking as such, but uses atomic / volatile operations, which implies a memory barrier (potentially very expensive and interfering with other possible optimizations).
Even if all the overhead of atomic operations can be eliminated by the JIT compiler in the single-threaded case, there is still overhead to decide which of the search buckets is admittedly a relatively quick calculation, but nonetheless, it cannot be excluded.
How to decide which implementation to use, the choice is probably simple.
If this is a static field, you will almost certainly want to use ConcurrentHashMap if testing does not show that it is a real performance killer. Your class has different expectations for thread safety from instances of this class.
If this is a local variable, then most likely, a HashMap is sufficient - if you do not know that references to the object may leak into another stream. By encoding the map interface, you allow yourself to easily change it if you find a problem.
If this is an instance field and the class was not designed to be thread safe, then document it as an unsafe thread and use the HashMap.
If you know that this instance field is the only reason the class is not thread safe and are willing to live with the limitations that promise the promise of thread safety, then use ConcurrentHashMap if testing does not show significant performance implications. In this case, you can let the class user select a thread-safe version of the object, possibly using another factory method.
In any case, document the class as a safe thread (or conditionally thread safe) so that the people who use your class know that they can use objects in multiple threads, and the people who edit your class know that they must maintain thread safety in the future.