Spark Streaming + Kinesis - Spark Session (Java)

SparkSessionand JavaStreamingContext with SparkConfcannot work together and influence the methods of others.

I want to use the application Spark( Java) to read data from Kinesis( StreamingContextexamples c only SparkConf), and also interact with the database (more conveniently with SparkSession).

Since SparkSessionthis is a new feature, and many of its functions are really convenient, I would like to switch to it, but all the Spark Kinesis examples that I can find use StreamingContextand SparkConf.

[documentation - integration of kinesisation with spark flow] [1]
[documentation - example of Spark Streaming] [2]

Can anyone share an example Spark Structured Streamingwith Kinesisusing SparkSession, if it can be done?

I'm still in college, so my knowledge of the subject is pretty depressed.

+4
source share

Source: https://habr.com/ru/post/1692573/


All Articles