First, I want to say that the behavior that you see can be achieved with a much smaller program, not to mention the calculations you do for circles. r.nextInt(250) for the positions of the circles would be enough to see the behavior, and it is much easier to see what happens. In addition, for debugging, I added a visible rectangle to the panel attached to the lines of the group layout, where you can see what happens:
final Rectangle background = new Rectangle(0, 0, 0, 0); root.layoutBoundsProperty().addListener(new ChangeListener<Bounds>() { @Override public void changed(ObservableValue<? extends Bounds> observable, Bounds oldValue, Bounds newValue) { background.setX(newValue.getMinX()); background.setY(newValue.getMinY()); background.setWidth(newValue.getWidth()); background.setHeight(newValue.getHeight()); } }); background.setFill(null); background.setStroke(Color.RED); pane.getChildren().add(background);
So what is going on here?
From the Group API:
Any transformation, effect, or state applied to a group will apply to all children of this group. Such transformations and effects will NOT be included in the framework of this group, however, if translations and effects are set directly for children of this group, they will be included in the framework of this group.
Looking at the result with the included weight:

You see that the boundaries of the group are greater than those inside. This is due to how the transformation is applied: the children of the group are transformed, but scaling is not considered to calculate the boundaries of the group. Thus, the group is in the panel where the unformed transformed circle boundaries are merged, then the transforms for the circles are applied.
Compare this statement with the result when scaling is disabled:

To summarize, this is a design, not a strange behavior, because the Group is always as big and positioned accordingly, where the limitations of its untransformed children are combined.
EDIT
If you want the nodes to be scaled in the position they are in and the group is not moving, I suggest directly scaling the children of the group. This implementation of your thread changes the scale of the circles for every 5 circles, but they remain in the same position:
new Thread(new Runnable() { private int count = 0; private double scale1 = .5; private double scale2 = .2; private double currentScale = scale1; @Override public void run() { final Random r = new Random(); try { while (true) { expand = expand * 1.1; Thread.sleep(700); Platform.runLater(new Runnable() { @Override public void run() { System.out.println(count); Circle c = new Circle(r.nextInt(250), r.nextInt(250), 30); c.setScaleX(currentScale); c.setScaleY(currentScale); root.getChildren().add(c); count++; if (count > 5){ count = 0; if (currentScale == scale1){ currentScale = scale2; } else { currentScale = scale1; } Iterator<Node> iterator = root.getChildren().iterator(); while (iterator.hasNext()) { Node next = iterator.next(); next.setScaleX(currentScale); next.setScaleY(currentScale); } } } }); } } catch (InterruptedException e) { e.printStackTrace(); } } }).start();