Download only the subset you need (possibly Latin), and not through the Google Fonts API. This may solve your problem.
Google Chrome seems to ignore the requested subsets, it gets all the available subsets along with its Unicode ranges and loads them according to the type of characters found throughout the page.
source share