Do I need to use setCache (true) in Node for the cache hint given by setCacheHint () to actually have any effect?
Yes.
The cache property is a hint that Node rendering should be cached (as an internal image) at all or not.
The cacheHint property is a hint for the expected Node transform system, so the cache operation can be optimized for these types of conversions (for example, cache hints rotate, scale, or speed, etc.).
If Node is not configured for caching at all, cacheHint does not matter.
Should calling setCache actually improve performance, i.e. frame rate for some time or most of the time?
Not necessary. JavaFX has a default frame rate limit of 60 frames per second, so if the performance is good enough that the frame rate is reached even without cache hints, you won't see any noticeable difference. This applies to many basic animations and transformations.
Even if the frame rate is not improved, a cache hint can make each conversion more efficient so that it is less intensive with a processor or GPU (usually with commercial quality).
There may be other things that have a much greater effect on frame rate. These things may have nothing to do with rendering the speed of the cached items (for example, lengthy work being done in the JavaFX application thread during the game loop or constantly changing the contents of the Node).
I used a combination of setCache(true) and setCacheHint(CacheHint.SPEED) in the small games that I wrote that included several animated nodes at the same time with several effects and transparency applied to the nodes. Settings greatly accelerated the work (Mac OS X, Macbook Air 2012, Java FX 2.2).
Instead of relying on hints for the rendering system, you can also manually take a snapshot of the Node tree and manually replace the Node tree with a snapshot. This snapshot technology is not always the best way, but it gives you an alternative if the tips in your case do not work very well.