All geek questions in one place

Apache Spark - Backend Servers

I developed a reporting application in PHP. The application is built with HTML, CSS, javascript libraries, graphic libraries (Highcharts) and MySQL for storing data. The user selects some parameters in the interface and clicks the "Submit" button. Then the PHP layer executes a bunch of required SQL queries and sends the json result back to the user interface, where charts and data tables are drawn.

The requirement now is to connect a large Apache Spark data solution to an existing application. I’ve been studying the last 2 weeks if I can somehow connect a PHP application using the REST API or some kind of Spark SQL driver to connect to the Spark SQL server and execute the same SQL set that I have now on Spark SQL . I have not chosen a solution yet. Now I started to learn Java-based technologies like Spring, others like Angularjs, Nodejs, other MVC environments to rewrite the project from scratch. I am not a big fan of java development since I am not a hardcore developer (I create some handy tools so that everything is done).

I read this - https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-PHP , but it looks like a separate spark installation. I am dealing with a huge cluster in my case.

I really appreciate any direction here.

+4
source share
3 answers

Yes, this can be done using the hive context and the intrinsically safe transaction server in the spark application.

you can run your spark application and do all the processing. After processing, if you use a Data frame, you just need to register it as a temporary table.

Now you can start the lean server from the spark application.

, jdbc PHP.

https://medium.com/@anicolaspp/apache-spark-as-a-distributed-sql-engine-4373e254e0f9#.ekc3cs28u

+4

, . Scala . .

+2

- , cloudera hortonworks?

In the case of Cloudera, you must use Impala and the corresponding JDBC drivers. In HDP, you must use the Spark Thrift Server with the appropriate JDBC drivers.

+1
source

Source: https://habr.com/ru/post/1655140/

More articles:


All Articles