"unresolved dependency" for Spark 2.0.1 on SBT

With my build.sbt

version := "1.0"
scalaVersion := "2.11.8"

// https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.1"

I am trying to run Spark in my sbt 0.13 project, but IntelliJ 2016.2.5 gives the error "unresolved dependency". What am I doing wrong?

There is no proxy server, and it works if I say "2.0.0" instead of "2.0.1", but it is not so nice ...

Also: it works in the CLI, but not in IntelliJ.

+4
source share
3 answers

SBT https://repo1.maven.org/maven2/ , , , Apache Spark. , , , , Ivy?

, maven:

resolvers ++= Seq(
  Resolver sonatypeRepo "public",
  Resolver typesafeRepo "releases",
)

UPDATE: MavenRepository ( , ), :

resolvers += "MavenRepository" at "https://mvnrepository.com/"
+1

build.sbt sp2, , ss uch as

C:\Users\marco\SparkExamples\src\main\scala\MachineLearningExamples.scala: 3: mllib org.apache.spark [error] import org.apache.spark.mllib.regression.LabeledPoint [] ^ [] C:\Users\marco\SparkExamples\src\main\scala\MachineLearningExamples.scala: 4: org.apache.spark.ml [error] import org.apache.spark.ml.classification._ [] ^

build.sbt.... , - sbt .

    name := "SparkExamples"
    version := "1.0"
    scalaVersion := "2.11.8"
    val sparkVersion = "2.0.1"


    libraryDependencies += "junit" % "junit" % "4.8" % "test"
    libraryDependencies ++= Seq("org.slf4j" % "slf4j-api" % "1.7.5",
                        "org.slf4j" % "slf4j-simple" % "1.7.5",
                        "org.clapper" %% "grizzled-slf4j" % "1.0.2")
    libraryDependencies += "org.apache.spark"%%"spark-core"   % sparkVersion 
    libraryDependencies += "org.apache.spark"%%"spark-streaming"   % sparkVersion 
    libraryDependencies += "org.apache.spark"%%"spark-mllib"   % sparkVersion 
    libraryDependencies += "org.apache.spark"%%"spark-streaming-flume-sink" % sparkVersion     
    libraryDependencies += "org.apache.spark"%%"spark-sql"   % sparkVersion 


    resolvers += "MavenRepository" at "https://mvnrepository.com/"

sparkVersion 1.6.0 ( ), fie

0

Mateusz Kubuszok . http://central.maven.org/maven2 , https://mvnrepository.com/.

resolvers  += "MavenRepository" at "http://central.maven.org/maven2"

, , .

0

Source: https://habr.com/ru/post/1659610/


All Articles