How to get 8 million Google Cloud entries using the bigquery API in Node.js?

Question

I am querying Google cloud data using Bigquery.

When I run the query, it returns about 8 million rows. But this causes an error:

Too big answer to return

How can I get all 8 million records, can someone help.

Answer 1

1. What is the maximum response size for a large request?

As mentioned in Quota-policy, it asks for the maximum response size: 128 MB compressed (unlimited when returning large query results)

2. " "?

, , allowLargeResults true .

, , , :

• .
• ORDER BY, TOP LIMIT . allowLargeResults, .
• , PARTITION BY.

, paginate , BigQuery Google Analytics, , 200, , Jobs:: getQueryResults maxResults int.

Update:

- , 128 .

BigQuery, 128 . , 128 . 2x? 10x? , , . , , , , . , , , , . , , , . ( , . 220)

Answer 2

,

" ", , ,

• ;
• " "

, .