Can we store relational data in hdfs

I am trying to convert an application that has a relational database as a backend. Can I store relational information in HDFS?

+6
source share
2 answers

Just for storage, you can store anything in HDFS. But that makes no sense. First of all, you should not think of Hadoop as a replacement for your RDBMS (which you are trying to do here). Both are designed for completely different purposes. Hadoop is not suitable for your transactional, relational, or real needs. It was designed to meet your needs for offline batch processing. So, it is better to analyze your use case correctly, and then freeze your decision.

As a suggestion, I would like to point you to Hive . It provides you with storage options over an existing Hadoop cluster. It also provides an SQL interface similar to your warehouse, which will make your life a lot easier if you come with SQL background. But then again, Hive is also a batch processing system and is not suitable if you need something in real time.

You can take a look at HBase , though, as suggested by abhinav. This is a database that can run on top of your Hadoop cluster and provides you with random access to your data for reading / writing in real time. But you must remember that this is NoSQL db. It does not comply with SQL terminology and conventions. Thus, you may find that it is a little alien. You may need to think about issues, such as storing your data in a new repository style (columnar), as opposed to storing the string styles of your RDBMS. Otherwise, it is not a problem to configure and use it.

NTN

+5
source

Any file can be saved in HDFS. But if you need a DB like SQL, you have to go for HBASE. If you directly store your data in HDFS, you cannot keep rationality.

0
source

Source: https://habr.com/ru/post/951023/


All Articles