Shuffle lines in spark frame

Question

I have a dataframe like this:

+---+---+
|_c0|_c1|
+---+---+
|1.0|4.0|
|1.0|4.0|
|2.1|3.0|
|2.1|3.0|
|2.1|3.0|
|2.1|3.0|
|3.0|6.0|
|4.0|5.0|
|4.0|5.0|
|4.0|5.0|
+---+---+

and I would like to shuffle all lines using a spark in scala.

How can I do this without returning to RDD?

Answer 1

You need to use the method orderByfor the data block:

import org.apache.spark.sql.functions.rand
val shuffledDF = dataframe.orderBy(rand())