I'm trying to create middleware that executes injection-safe dynamic SQL queries using Spark, with Cassandra as the storage engine. Query results must be saved in the newly created Cassandra table.
Spark SQL does not provide any functions for safely binding parameters, so I thought about using a lean JDBC server and JDBC interface.
Inserting data into an external table from Hive is done by running
CREATE EXTERNAL TABLE <table_name>... STORED BY...
However, when I try to execute this statement through a lean server, I always get the following error:
java.sql.SQLException: org.apache.spark.sql.catalyst.parser.ParseException: Operation not allowed: STORED BY(line 1, pos 31)
After some web digging, I could find that the reason was that the thrifty server used Spark SQL and that Hive support was disabled by default on Spark SQL. Then the general recommendation is to manually create a SparkSession and use enableHiveSupport ().
The problem is that I do not want to do this, because I cannot securely bind parameters to Spark SQL. I want to use only the JDBC interface. Do you know if there is a way to enable Hive support when starting a lean server?
If there is no way, can someone recommend a safe way to bind a parameter?
I am using Spark 2.1.0
EDIT
In fact, I also failed to create an external table through SparkSQL with a manually created SparkSession, even created with enableHiveSupport() . Still getting the same exception:
org.apache.spark.sql.catalyst.parser.ParseException: Operation not allowed: STORED BY(line 1, pos 42)