All geek questions in one place

Uploading data to the Titan database

I have a log data set in the form of flat files from which I want to create a graph (based on the information in the log) and upload it to the Titan database. This data is several gigabytes. I am exploring Faunus and BatchGraph bulk upload options (which I read about at https://github.com/thinkaurelius/titan/wiki/Bulk-Loading ). The tab shows the log data. I need a bit of processing on each line of the file to form the nodes and edges of the graph that I mean. Will Faunus / BatchGraph serve in this case? If so, in what format should my input file be used for these tools to work? If not, does the BluePrints API use the path? Any resources that you can share with your offer are greatly appreciated since I am new. Thank!

+4
source share
1 answer

To answer your question in a simple way, I think you will want to use Faunus to upload your data. I would recommend first cleaning up and converting your data using external tools, if possible. A tab delimiter is a great format, but the way you prepare this file can affect download performance (for example, sometimes just sorting the data in the right way can provide more speed).

A more complete answer lies in these two resources. They should help you choose the approach:

http://thinkaurelius.com/2014/05/29/powers-of-ten-part-i/ http://thinkaurelius.com/2014/06/02/powers-of-ten-part-ii/

- , , - 100K 1M. BatchGraph API- Blueprints, . , . , . , .

+6

Source: https://habr.com/ru/post/1548007/

More articles:


All Articles