GWT is JavaScript, and JavaScript is the standard . If someone wanted to dramatically change JavaScript, they would love it. GWT is also open source , so you can tell what they do and how they do it if you have the time and inclination to investigate.
Silverlight is closed and proprietary , and, as such, completely at the whim of a single Microsoft company. If you trust Microsoft not to dramatically change Silverlight’s internal work in such a way as to adversely affect your development, this may not matter. But I personally do not.
In addition, as you said, Silverlight requires the use of a plug-in, which will annoy users not only the first time they have to download the plug-in, but anytime they need to update it to use new features forever.