All geek questions in one place

How can I extract data from OData API in SQL?

I am interested in what the community offers for retrieving data from the OData API in SQL 2008 R2. I need to create a night job that imports data into SQL. Should I create a simple console application that iterates through the OData API and imports into SQL? Or can I create some kind of SQL Server BI application? Or is there a better way to do this?

+4
source share
1 answer

It will be slow. OData is not a bulk API. It is designed so that clients can access individual objects and move between them, in most cases break pages into some filtered lists.

Retrieving the entire dump through OData will not make anyone happy. The owner of the OData API will need to investigate who does all these nightly crawls on his API and discovers that you are the one and probably disconnected you. On the other hand, you will find that OData is not an efficient mass transport format, but marshaling HTTP encoded objects back and forth is not the best way to spend your bandwidth. And crawling the entire database every time, unlike just detecting the deltagrams from the last crawl, will only work until the database reaches such a critical size S that the update takes longer than the frequency you combine!

Also, if this is not your data, it is highly likely that terms of use for the OData API explicitly prevent such massive crawls.

Get a data dump, archive it and copy using FTP.

+3
source

Source: https://habr.com/ru/post/1332842/

More articles:


All Articles