All geek questions in one place

Scala + SBT - How to set up reference.conf for the shaded Akka library

TL DR

I am trying to obscure the version of the library akkaand associate it with my application (in order to be able to start the server spray-canin the version CDH 5.7 Spark 1.6). The shading process puts akkathe default configuration and after manually providing a separate version akka reference.conffor the hatched one akka, it still looks like the 2 versions are somehow messed up.

Are shading akkaversions of the problem known ? What am I doing wrong?

Background

I have an application Scala/ Sparkrunning offline Spark 1.6.1. The application starts the spray-canhttp server using the spray 1.3.3one for which it is required akka 2.3.9( Spark 1.6.1standalone includes compatible akka 2.3.11).

I'm trying to move an application to the new Clouderaon the basis of Sparka cluster that is running version CDH 5.7 Spark 1.6. The problem is what is Spark 1.6in CDH 5.7contact with akka 2.2.3, which is not enough for proper operation spray 1.3.3.

Attempt to solve

, akka 2.3.9 . - akka , reference.conf, . - sbt-assembly , akka .

, akka :

ShadeRule.rename("akka.**" -> "akka_2_3_9_shade.@1")
    .inLibrary("com.typesafe.akka" % "akka-actor_2.10" % "2.3.9")
    .inAll

reference.conf , akka original reference.conf, "akka" "akka_2_3_9_shade".

, , Spark -provided akka - akka, :

Exception in thread "main" java.lang.IllegalArgumentException: Cannot instantiate MailboxType [akka.dispatch.UnboundedMailbox], defined in [akka.actor.default-mailbox], make sure it has a public constructor with [akka.actor.ActorSystem.Settings, com.typesafe.config.Config] parameters
    at akka_2_3_9_shade.dispatch.Mailboxes$$anonfun$1.applyOrElse(Mailboxes.scala:197)
    at akka_2_3_9_shade.dispatch.Mailboxes$$anonfun$1.applyOrElse(Mailboxes.scala:195)
    at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
    at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185)
    at scala.util.Try$.apply(Try.scala:161)
    at scala.util.Failure.recover(Try.scala:185)
    at akka_2_3_9_shade.dispatch.Mailboxes.lookupConfiguration(Mailboxes.scala:195)
    at akka_2_3_9_shade.dispatch.Mailboxes.lookup(Mailboxes.scala:78)
    at akka_2_3_9_shade.actor.LocalActorRefProvider.akka$actor$LocalActorRefProvider$$defaultMailbox$lzycompute(ActorRefProvider.scala:561)
    at akka_2_3_9_shade.actor.LocalActorRefProvider.akka$actor$LocalActorRefProvider$$defaultMailbox(ActorRefProvider.scala:561)
    at akka_2_3_9_shade.actor.LocalActorRefProvider$$anon$1.<init>(ActorRefProvider.scala:568)
    at akka_2_3_9_shade.actor.LocalActorRefProvider.rootGuardian$lzycompute(ActorRefProvider.scala:564)
    at akka_2_3_9_shade.actor.LocalActorRefProvider.rootGuardian(ActorRefProvider.scala:563)
    at akka_2_3_9_shade.actor.LocalActorRefProvider.init(ActorRefProvider.scala:618)
    at akka_2_3_9_shade.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
    at akka_2_3_9_shade.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
    at akka_2_3_9_shade.actor.ActorSystemImpl._start(ActorSystem.scala:616)
    at akka_2_3_9_shade.actor.ActorSystemImpl.start(ActorSystem.scala:633)
    at akka_2_3_9_shade.actor.ActorSystem$.apply(ActorSystem.scala:142)
    at akka_2_3_9_shade.actor.ActorSystem$.apply(ActorSystem.scala:109)
    at akka_2_3_9_shade.actor.ActorSystem$.apply(ActorSystem.scala:100)
    at MyApp.api.Boot$delayedInit$body.apply(Boot.scala:45)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
    at scala.App$class.main(App.scala:71)
    at MyApp.api.Boot$.main(Boot.scala:28)
    at MyApp.api.Boot.main(Boot.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassCastException: interface akka_2_3_9_shade.dispatch.MailboxType is not assignable from class akka.dispatch.UnboundedMailbox
    at akka_2_3_9_shade.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:69)
    at akka_2_3_9_shade.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:66)
    at scala.util.Try$.apply(Try.scala:161)
    at akka_2_3_9_shade.actor.ReflectiveDynamicAccess.getClassFor(DynamicAccess.scala:66)
    at akka_2_3_9_shade.actor.ReflectiveDynamicAccess.CreateInstanceFor(DynamicAccess.scala:84)
    ... 34 more

Boot.scala :

[45]    implicit val system = ActorSystem()
...
[48]    val service = system.actorOf(Props[MyAppApiActor], "MyApp.Api")
...
[52]    val port = config.getInt("MyApp.server.port")
[53]    IO(Http) ? Http.Bind(service, interface = "0.0.0.0", port = port)
+4
2

, .

akka ( ) , . , akka/actor/ActorSystem.scala.

, sbt-assembly / .

, sbt-assembly shading. , , akka, , ActorSystem, , :

ConfigException$Missing: No configuration setting found for key 'akka_2_3_9_shade'

, ( , , akka_spray_shade.conf), :

  • akka original reference.conf, akka , akka_2_3_9_shade. ( )
  • akka original reference.conf, akka , akka_2_3_9_shade akka akka_2_3_9_shade. ( , sbt-assembly)
  • spray original reference.conf, akka , akka_2_3_9_shade. (, spray akka)

ActorSystem Boot.scala:

val akkaShadeConfig = ConfigFactory.load("akka_spray_shade")
implicit val system = ActorSystem("custom-actor-system-name", akkaShadeConfig)
+6

.

, akka_spray_shade.conf. application.conf, ActorSystem, : ActorSystem("custom-actor-system-name") ActorSystem("custom-actor-system-name", ConfigFactory.load("application")).

+1

Source: https://habr.com/ru/post/1661455/

More articles:


All Articles