All geek questions in one place

Error starting Spark in Intellij: "the apache object is not a member of the org package"

I run the Spark program on Intellij and get the following error: "the apache object is not a member of the org package."

I used this import statement in code:

import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf

The above import statement also fails to run at the sbt prompt. The corresponding lib seems to be missing, but I'm not sure how to copy the same and in which path.

+7
source share
1 answer

Make sure the SBT has entries like this:

 scalaVersion := "2.11.8" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % "2.1.0", "org.apache.spark" %% "spark-sql" % "2.1.0" )

Then make sure that IntelliJ is aware of these libraries, either by turning on “automatic import” or by doing it manually by clicking the “Update” button on the SBT panel.

+7
source

Source: https://habr.com/ru/post/1266154/

More articles:


All Articles