Your decision is quite reasonable. I have an Excel COM add-in that does something very similar. In this code, I install Application.Handle in the DLL as the window handle of the Excel main window. It is like what you are doing.
The problem is that you need to correctly set the ownership of the window. You need a property chain to return to the main form of the application. Forms in a DLL do not know what the main form is, and therefore you must provide this knowledge.
Note that I'm talking about the window owner concept used by Windows, and not about the VCL owner concept, which is completely different. In VCL terminology, this is called a pop-up parent, and you can solve your problem by explicitly indicating that the pop-up parent window of the DLL form is the main form. The relevant properties are PopupMode and PopupParent. For forms that are in the main application, VCL will naturally make them the pop-up parent of the main form.
However, speaking of explicitly configuring the pop-up parent, I would like to emphasize that your current solution is simpler and more convenient.
What both of these solutions do is make sure that all auxiliary forms belong to the main form. This means that these forms are always on top of the main form. This means that auxiliary forms will be minimized if the main form is minimized. Read about the windows available here: Window Properties .
By the way, if you used runtime packages rather than DLLs, the code in the package would be associated with the same VCL as the main form. Thus, the packaged code will be able to see the main form and set the window owner accordingly. This is certainly one of the benefits of using packages. Of course, this may well be a good reason why you need to use DLLs, not packages.