Although the documentation says that debugging is not supported, you can actually attach a debugger and debug your code (I think this may mean that you cannot go through the interactive code or break it).
All you have to do is go to Debug> AttachToProcess and then view the list for InteractiveHost.exe. This can be a bit more complicated if you are using multiple instances of Visual Studio, as there may be a different process for each of them. But if you attach a debugger, you will certainly get to breakpoints in your own code that you used to initialize the interactive session.