Spark dataset conversion to RDD

I have a Dataset [String] and you need to convert to RDD [String]. How?

Note. I recently switched from spark 1.6 to spark 2.0. Some of my clients were expecting RDD, but now Spark is giving me Dataset.

+7
source share
2 answers

As stated in the scala API documentation , you can call .rddin your dataset:

val myRdd : RDD[String] = ds.rdd
+18
source

A dataset is a strongly typed Dataframe, so both Dataset and Dataframe can use .rdd to convert to RDD.

0
source

Source: https://habr.com/ru/post/1650356/


All Articles