What are the required options for downloading an excel file?

I downloaded the excel file from S3 using the syntax below, but I am wondering what parameters need to be set here.

Why is it necessary to install all of the options below to download an excel file? None of these options are required to download other types of files, such as csv, del, json, avro, etc.

val data = sqlContext.read.
format("com.crealytics.spark.excel").
option("location", s3path).
option("useHeader", "true").
option("treatEmptyValuesAsNulls", "true").
option("inferSchema","true").
option("addColorColumns", "true").
load(path)

I get the following error if any of the above parameters (except the location) is not set:

sqlContext.read.format("com.crealytics.spark.excel").option("location", s3path).load(s3path)

Error message:

Name: java.lang.IllegalArgumentException
Message: Parameter "useHeader" is missing in options.
StackTrace:   at com.crealytics.spark.excel.DefaultSource.checkParameter(DefaultSource.scala:37)
          at com.crealytics.spark.excel.DefaultSource.createRelation(DefaultSource.scala:19)
          at com.crealytics.spark.excel.DefaultSource.createRelation(DefaultSource.scala:7)
          at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:345)
          at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
          at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:132)
          at $anonfun$1.apply(<console>:47)
          at $anonfun$1.apply(<console>:47)
          at time(<console>:36)
+4
source share
1 answer

Most parameters spark-excelare required, with the exception of userSchemaand sheetName.

DataSource, .

, , .

+3

Source: https://habr.com/ru/post/1678753/


All Articles