NoSuchMethodError exception exception in uber jar

Question

These are the dependencies inside - this is my file pom.xml. When I run the driver from my maven project as a Scala application, it works fine. But when I create jar-with-dependencyand try to run my project throughspark-submit

spark-submit --class package.signature.Driver --master local[*] /path/to/my/jar-with-dependencies.jar

I get the following exception.

Exception in thread "main" java.lang.NoSuchMethodError :. Org.apache.http.conn.ssl.SSLConnectionSocketFactory (Ljavax / network / SSL / SSLContext; [Ljava / languages ​​/ String; [Ljava / languages ​​/ String; Ljavax / network / SSL / HostnameVerifier;) V, in com.gargoylesoftware. htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory. (HtmlUnitSSLConnectionSocketFactory.java:125) in com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.buildSSLSocketFactory (HtmlUnitSSLConnectionSocketFactory.java:112) in com.gargoylesoftware.htmlunit.HttpWebConnection.configureHttpsScheme (HttpWebConnection.java:597) in com.gargoylesoftware.htmlunit. HttpWebConnection.createHttpClient (HttpWebConnection.javaPoint32) in com.gargoylesoftware.htmlunit.HttpWebConnection.getHttpClientBuilder (HttpWebConnection.java:494) in com.gargoylesoftware.htmlWitconnection.ttgetResponse (HttpWebConnection.java:158) in com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection (WebClient.java:1321) in com.gargoylesoftware.htmlunit.WebClient.loadWebResponj.gt.client.wlogle.client.client.client.jt .getPage (WebClient.javahaps46) in com.gargoylesoftware.htmlunit.WebClient.getPage (WebClient.java:415) in org.openqa.selenium.htmlunit.HtmlUnitDriver.get (HtmlUnitDriver.javahaps41) in org.open selenium.htmlunit.HtmlUnitDriver.get (HtmlUnitDriver.java∗30)getPage (WebClient.java:415) in org.openqa.selenium.htmlunit.HtmlUnitDriver.get (HtmlUnitDriver.java∗41) in org.openqa.selenium.htmlunit.HtmlUnitDriver.get (HtmlUnavaDriver)getPage (WebClient.java:415) in org.openqa.selenium.htmlunit.HtmlUnitDriver.get (HtmlUnitDriver.java∗41) in org.openqa.selenium.htmlunit.HtmlUnitDriver.get (HtmlUnavaDriver)

.getPageSource

import org.openqa.selenium.htmlunit.HtmlUnitDriver
import java.util.concurrent.TimeUnit
import scala.concurrent.duration._
import org.apache.spark.Logging

object Selenium extends Logging {
  val driver = new HtmlUnitDriver(false)
  val implicitWaitTimeout = 30 // seconds
  driver.manage.timeouts.implicitlyWait(implicitWaitTimeout, TimeUnit.SECONDS)
  def getPageSource(urlPage: String): String = {
    driver.get(urlPage)
    driver.getPageSource
  }
}

, . apache-httpcomponent. .

Answer 1

, , , , , , maven , , Ctrl + Shift + T, , , , pom.xml .jar, .

Answer 2

- , selenium-htmlunit-driver ( com.gargoylesoftware.htmlunit ) org.apache.httpcomponents.httpclient v. 4.4 ( , SSLConnectionSocketFactory, 4.4 , 4.4, . ). , selenium-htmlunit-, , 4.5.1. :

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.1</version>
</dependency>

JAR. , , , httpclient-x.x.x.jar .