SparkSessionand JavaStreamingContext with SparkConfcannot work together and influence the methods of others.
I want to use the application Spark( Java) to read data from Kinesis( StreamingContextexamples c only SparkConf), and also interact with the database (more conveniently with SparkSession).
Since SparkSessionthis is a new feature, and many of its functions are really convenient, I would like to switch to it, but all the Spark Kinesis examples that I can find use StreamingContextand SparkConf.
[documentation - integration of kinesisation with spark flow] [1]
[documentation - example of Spark Streaming] [2]
Can anyone share an example Spark Structured Streamingwith Kinesisusing SparkSession, if it can be done?
I'm still in college, so my knowledge of the subject is pretty depressed.
source
share