I understand that you are asking for Java, but my experience with this was with C ++; Java equivalent syntax should work, minus customizability.
Take a look at these examples for selection and scaling. In addition, if you set the style of interaction with the image, the mouse wheel should cause scaling anywhere in the cursor. You probably do not want to literally do what you requested, but rather either scale the rubber band, or have a wheel for scaling. A click should do something, and not just change the look.
http://www.vtk.org/Wiki/VTK/Examples/Cxx/Images/PickingAPixel2
http://www.vtk.org/Wiki/VTK/Examples/Cxx/Interaction/RubberBandZoom
source share