Webmaster APIs - Quota Limits

Question

We are trying to load page data for sites using the API.NET Webmaster client library by calling WebmastersService.SearchAnalytics.Query(). To do this, we use Batching and send approx. 600 requests in one batch. However, most of them fail with the "Quota Exceeded" error. The number of failures changes every time, but only about 10 out of 600 work (and it changes where they are in the party). The only way to make it work is to reduce the batch size to 3 and wait 1 second between each call.

According to the development console, our daily quota is set at 1,000,000 (and we have 99% left), and our user limit is set at 10,000 queries / second / user.

We return an error:

Quota exceeded [403] Errors [Message [Quota exceeded] Location [-] Reason [quotaExceeded] Domain [usageLimits]]

Is there another quota that applies? What does “Domain [Usage Limits]” mean is the domain to which we request page data, or is it our user account?

There is still a problem if we run each request separately, if we do not expect 1 second between each call. Due to the number of sites and the number of pages that we need to download data for this, this is actually not an option.

, , , - 1000, , Google, , , , ( ) .

1

. , , ; o)

using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using Google.Apis.Util.Store;
using Google.Apis.Webmasters.v3;
using Google.Apis.Webmasters.v3.Data;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            new Program().Run().Wait();
        }

        private async Task Run()
        {
            List<string> pageUrls = new List<string>();
            // Add your page urls to the list here

            await GetPageData("<your app name>", "2015-06-15", "2015-07-05", "web", "DESKTOP", "<your domain name>", pageUrls);
        }

        public static async Task<WebmastersService> GetService(string appName)
        {
            //if (_service != null)
            //    return _service;
            //TODO: - look at analytics code to see how to store JSON and refresh token and check runs on another PC
            UserCredential credential;
            using (var stream = new FileStream("c:\\temp\\WMT.json", FileMode.Open, FileAccess.Read))
            {
                credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(
                    GoogleClientSecrets.Load(stream).Secrets,
                    new[] { Google.Apis.Webmasters.v3.WebmastersService.Scope.Webmasters },
                    "user", CancellationToken.None, new FileDataStore("WebmastersService"));
            }

            // Create the service.
            WebmastersService service = new WebmastersService(new BaseClientService.Initializer()
            {
                HttpClientInitializer = credential,
                ApplicationName = appName,
            });

            //_service = service;
            return service;

        }

        private static async Task<bool> GetPageData(string appName, string fromDate, string toDate, string searchType, string device, string siteUrl, List<string> pageUrls)
        {
            // Get the service from the initial method
            bool ret = false;

            WebmastersService service = await GetService(appName);
            Google.Apis.Requests.BatchRequest b = new Google.Apis.Requests.BatchRequest(service);

            try
            {
                foreach (string pageUrl in pageUrls)
                {
                    SearchAnalyticsQueryRequest qry = new SearchAnalyticsQueryRequest();
                    qry.StartDate = fromDate;
                    qry.EndDate = toDate;
                    qry.SearchType = searchType;
                    qry.RowLimit = 5000;
                    qry.Dimensions = new List<string>() { "query" };
                    qry.DimensionFilterGroups = new List<ApiDimensionFilterGroup>();
                    ApiDimensionFilterGroup filterGroup = new ApiDimensionFilterGroup();
                    ApiDimensionFilter filter = new ApiDimensionFilter();
                    filter.Dimension = "device";
                    filter.Expression = device;
                    filter.Operator__ = "equals";

                    ApiDimensionFilter filter2 = new ApiDimensionFilter();
                    filter2.Dimension = "page";
                    filter2.Expression = pageUrl;
                    filter2.Operator__ = "equals";

                    filterGroup.Filters = new List<ApiDimensionFilter>();
                    filterGroup.Filters.Add(filter);
                    filterGroup.Filters.Add(filter2);
                    qry.DimensionFilterGroups.Add(filterGroup);

                    var req = service.Searchanalytics.Query(qry, siteUrl);
                    b.Queue<SearchAnalyticsQueryResponse>(req, (response, error, i, message) =>
                    {
                        if (error == null)
                        {
                            // Process the results
                            ret = true;
                        }
                        else
                        {
                            Console.WriteLine(error.Message);
                        }

                    });
                    await b.ExecuteAsync();
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Exception occurred getting page  stats : " + ex.Message);
                ret = false;
            }

            return ret;
        }        
    }
}

program.cs Google.Apis.Webmasters.v3 nuget. wmt.json c:\temp, . 5 URL- pageUrls, .

Answer 1

, . , (1/), (20/). , , rateLimitExceeded , quotaExceeded. , , Google ( , , , 20/), .