All geek questions in one place

Unresolved dependency on Spark 1.2.0 build

I am trying to create Spark 1.2.0 on ubuntu, but I am getting dependency problems.

I basically download files by unpacking the folder and run sbt / sbt / assembly

  • sbt = 0.13.6

  • scala = 2.10.4

sbt.ResolveException: unresolved dependency: org.apache.spark#spark- network-common_2.10;1.2.0: configuration not public in org.apache.spark#spark-network-common_2.10;1.2.0: 'test'. It was required from org.apache.spark#spark-network-shuffle_2.10;1.2.0 test
+6
source share
1 answer

This sbt issue seems to explain this: it would be the result of trying to get a test dependency if the same version were allowed from the Maven public repository.

A workaround would be to use git SHA versions or SNAPSHOT to not final build this test dependency, but we won’t know more if we don’t get an idea of ​​how you got into a bad ivy state. / p>

TL DR: try clearing the cache of spark artifacts before building.

Edit: this is fixed in sbt 0.13.10-RC1 https://github.com/sbt/sbt/pull/2345 Update

0
source

Source: https://habr.com/ru/post/981390/

More articles:


All Articles