Eclipse does not have sbt plugin
although sbt has an Eclipse plugin
this is a workaround, but not the right solution.
this means that eclipse does not understand build.sbt, as it does for pom.xml or build.gradle, so if we change the dependency in our eclipse project in the build.sbt file, eclipse will not understand it and will not change the dependency in the dependency project . So you should take the following steps every time you change any dependency.
The workaround is as follows
Step 1: Download and install sbt from https://www.scala-sbt.org/release/docs/Setup.html for Windows, it's quite simple https://piccolo.link/sbt-1.2.8.msi is pretty simple forward
Step 2. Create a folder, say D: \ sbt \ edge. Now create the build.sbt file here. You can use the following as content, modify it accordingly.
name := "edge" version := "0.1" scalaVersion := "2.11.8" val sparkVersion="2.4.0" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion, "org.apache.spark" %% "spark-sql" % sparkVersion, "org.apache.spark" %% "spark-mllib" % sparkVersion % "runtime", "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided", "org.apache.spark" %% "spark-hive" % sparkVersion % "provided", "org.apache.spark" %% "spark-catalyst" % sparkVersion % Test, "org.apache.spark" %% "spark-graphx" % sparkVersion, "org.apache.spark" %% "spark-repl" % sparkVersion % "provided", "org.apache.spark" %% "spark-yarn" % sparkVersion, "org.apache.spark" %% "spark-mllib-local" % sparkVersion, //"org.apache.spark" %% "spark-streaming-kafka" % "1.6.3", //"org.apache.spark" %% "spark-streaming-twitter" % "1.6.3", "ch.qos.logback" % "logback-classic" % "1.1.3" )
Now create these subfolders src and main as → D: \ sbt \ spark \ src \ main
Step 3: Open CMD / PowerShell, go to D: \ sbt \ edge and run the "sbt package"
Step 4: Go to .sbt in your home directory and to the plugins folder of the correct version in my case C: \ Users \ xxxx.sbt \ 1.0 \ plugins, if you already have the plugins.sbt file, add the following line addSbtPlugin ("com .typesafe.sbteclipse "%" sbteclipse-plugin "%" 5.2.4 "), if not, create and add a line.
Step 5: Return to CMD / PowerShell, run sbt eclipse. This will create the necessary files for the eclipse project. Step 6. Open the Eclipse workspace and import this project as existing.