SparkSQL column chart in group by condition

Question

I am moving the query from Hive to SparkSQL, but having run into one problem with the Map column.

My request

spark.sql(select col1,col2,my_map,count(*) from table group by col1,col2,my_map)

The error I get is

`my_map` cannot be used as a grouping expression because its data type map<string,string> is not an orderable data type.;

The keys in my_map always change. I tried to use the deprecated HiveContext, but that did not help. Is there a workaround for this?

Thank!

Answer 1

The answer is in the error response. You need to make my_map into an ordered data type! :)