How to use SQL query to define table in dbtable?

In JDBC for other databases, I found the following parameter explanation dbtable:

The JDBC table to be read. Note that you can use everything that is valid in the FROM clause of the SQL query. For example, instead of a full table, you can also use a subquery in parentheses.

When I use the code:

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
  url "jdbc:postgresql:dbserver",
  dbtable "mytable"
)

everything works fine, but the following:

 dbtable "SELECT * FROM mytable"

leads to an error:

enter image description here

What's wrong?

+3
source share
2 answers

dbtable SELECT, , SQL-. , :

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
    url "jdbc:postgresql:dbserver",
    dbtable "(SELECT * FROM mytable) tmp"
);

:

SELECT * FROM (SELECT * FROM mytable) tmp WHERE 1=0
+12

Scala

val checkQuery = "(SELECT * FROM " + inputTableName + " ORDER BY " + columnName + " DESC LIMIT 1) AS timetable"

val timeStampDf = spark.read.format("jdbc").option("url", url).option("dbtable", checkQuery).load()

.

0

Source: https://habr.com/ru/post/1673839/


All Articles