Using Storm in Cloudera

I wanted to use Storm, which is available with the installation of Hortonworks 2.1, but in order to avoid installing Hortonworks in addition to installing Cloudera (which has Spark), I tried to find a way to use Storm in Cloudera.

If both Storm and Spark can be used on the same platform, this will save the additional resources needed to install Cloudera and Hortonworks on the machine.

+6
source share
5 answers

You can use the storm with the installation of Cloudera. You will have to install it yourself and maintain it as such. It will not be part of the Cloudera stack, but this should not stop you from using it with Hadoop if you need it.

+5
source

You can use Storm on any vendor platform. However, managing a thunder cluster is what you need to consider. The storm is not part of the CDH distribution. Cloudera Manager does not manage the life cycle of storm services and configurations, nor does it track the storm cluster unless you want to write the Clouderea Manager extension yourself. In contrast, if you choose a provider such as HDP, the Ambari HDP management tool provides all of the management features listed above.

If you have a streaming project on CDH, you should first consider Apache Spark, as it provides the same programming model for batch and streaming processing. You do not need to learn the new API. However, Apache Spark streaming is a micro-package. Thus, in applications requiring real-time processing with low latency, Storm is more suitable.

+1
source

You can use Storm next to Cloudera.

0
source

All of the above is true, but why do you need it?

Spark includes Spark Streaming, which allows you to process data processing and process streams / events using a single API. Spark / Streaming is already inside CDH.

So why load yourself with two different APIs?

0
source

You can install Apache Storm on the Cloudera virtual machine. For basic installation and test run, follow the link below:

https://github.com/vrmorusu/StormOnClouderaVM/wiki/Apache-Storm-on-Cloudera-VM

This should help you get started developing your Storm applications on the Cloudera VM.

0
source

Source: https://habr.com/ru/post/972813/


All Articles