Context: I am working on a Spark test that writes data to InfluxDB using this library . Here is the environment.
- Scala 2.11.8
- Spark 2.1.0 (Docking Autonomous Cluster)
corresponding dependencies:
"org.apache.spark" %% "spark-core" % "2.1.0" % "provided", "org.apache.spark" %% "spark-streaming" % "2.1.0" % "provided", "org.apache.spark" %% "spark-streaming-kafka-0-8" % "2.1.0", "com.paulgoldbaum" %% "scala-influxdb-client" % "0.5.2" // which uses "org.asynchttpclient" % "async-http-client" % "2.0.24"
Everything compiles and works fine on my local computer, but when I submit the assembly to the Spark cluster, I get this error in the driver:
Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:58) at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala) Caused by: java.lang.IllegalAccessError: tried to access field io.netty.handler.ssl.JdkSslContext.SUPPORTED_CIPHERS from class io.netty.handler.ssl.NettySslPackageAccessor at io.netty.handler.ssl.NettySslPackageAccessor.jdkSupportedCipherSuites(NettySslPackageAccessor.java:24) at org.asynchttpclient.config.AsyncHttpClientConfigDefaults.defaultEnabledCipherSuites(AsyncHttpClientConfigDefaults.java:85) at org.asynchttpclient.DefaultAsyncHttpClientConfig$Builder.<init>(DefaultAsyncHttpClientConfig.java:635) at org.asynchttpclient.DefaultAsyncHttpClient.<init>(DefaultAsyncHttpClient.java:67) at com.paulgoldbaum.influxdbclient.HttpClient.<init>(HttpClient.scala:21) at com.paulgoldbaum.influxdbclient.InfluxDB$.connect(InfluxDB.scala:16) ...
The problem disappears if I delete the code for writing to InfluxDB.
What I found out after some inspection is that the io.netty.handler.ssl.NettySslPackageAccessor class actually belongs to the async-http-client library. There seems to be a hack class to access the protected member in io.netty.handler.ssl.JdkSslContext .
I mixed up this issue for several days. The solution I got to get it working overrides the async-http-client an earlier version that does not include a violation code.
dependencyOverrides ++= Set("org.asynchttpclient" % "async-http-client" % "2.0.12")
Question: Why does IllegalAccessError occur only in the cluster, and not in my local launch? Is there a better way to solve this problem?
If my SBT can compile a fine, then there shouldn't be such an IllegalAccessError , so this means that there are differences between my local code and the cluster code, which is probably provided spark dependencies, but this is the same version as the cluster.
I kind of leave things as they are, but it would be better if newer versions were used. Or at least I want to understand why this problem arises and avoid it in the future.