What is the practical difference between google datatore nosql and google bigquery sql server?

Question

I want to know how to evaluate one instrument over another.

My main concern is as follows:

In google datastore we define 'kind'. Each “entity” has “properties”.
The data warehouse database then uses these properties to index the data for a future query. The query itself uses almost the same idea in SQL, although it differs from the syntax to filter the data and find what we want.
If you index each property, the index metadata will be even larger than the actual data.

Google bigquery uses this dialect of SQL. And it is fully manageable, so users do not need to worry about the problem of scaling.

So my question is: why use the nosql datastore, spend so many computational cycles and storage for indexes so that we can simply query it, as we naturally could in a bigquery SQL query database?

Please share your assessment process with me. Share with me what is lacking in my understanding. Thanks.

Answer 1

Both services solve different problems.

Datastore is designed to support transactional workloads such as the backend for a web application. It is optimized for small transactions that read or write a limited number of lines per operation, with strong consistency guarantees.

BigQuery is designed for analytic workloads. It is only append and optimized for queries that scan / filter / aggregate entire data tables to get answers from your data.

So the real question is: what problem are you trying to solve? If you are creating a web application and need a backend, use Datastore. If you have piles of data that you want to query to get answers about how your business works, use BigQuery.

Finally, if you want to do both, you can use the Datastore for your application and then export to BigQuery for analysis: https://cloud.google.com/bigquery/loading-data-cloud-datastore