Please refer to neighbor's answer in What can I call unmanaged C ++ Dll
In general, the strong name for the assembly is only a checksum that no one has changed / cracked / entered into your code (the injection can be something like a virus or a Trojan application). This is not very secure code protection, so it does not take a lot of time to complete all assembly tests. Of course, a signed code will take a little longer to load than a simple build.
PInvoke, on the other hand, is like a web service call, where the .Net code doesn't mind if this code is safe or not. There may be delays here only if you have a lot of arguments, and the argument types come with some third-party structures from some large signed assemblies. And here you can see slight time delays, because this large assembly is loaded into memory, types are checked, and PInvoke is done.
Perhaps you should try to move all classes and structures from many assemblies to one and use PInvoke there. Thus, in this case, you will not need to download many assemblies.