Creating JARs Storm compile time only in Gradle project

Question

I am trying to create a Gradle project that contains a Storm project. To run this project on Storm, I must first create a JAR file and let Storm run my topology, for example.

storm jar myJarFile.jar com.mypackage.MyStormMainClass 

I have problems because Gradle, by default, includes Storm dependencies both at compile time and at runtime. This throws the following exception:

 Exception in thread "main" java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar. 

This exception is really helpful and tells us the root cause of the problem. The solution is to include Storm dependencies when compiling with Gradle, but not when creating the final JAR file.

Does anyone know how to solve this? Other posts in StackOverflow did not solve the problem. If you embed code, make sure it really works.

Thanks!

Answer 1

Below I quote my response to a similar thread to the storm users mailing list .

In my case, I decided to use the fatJar plugin for Gradle for this purpose.

 buildscript { repositories { mavenCentral() mavenRepo url: "http://clojars.org/repo" } dependencies { // see https://github.com/musketyr/gradle-fatjar-plugin classpath 'eu.appsatori:gradle-fatjar-plugin:0.2-rc1' } } // When using this plugin you can build a fat jar / uberjar with 'gradle fatJar' apply plugin: 'fatjar' dependencies { compile 'storm:storm:0.8.2', { ext { fatJarExclude = true } } } 

You build a fat jar through:

 $ gradle clean fatJar 

Best Michael

PS: For which I also wrote a blog about how to solve this problem, as described above, in Launching a Cluster Cluster <-> w630>, This post contains additional pieces of information and gotchas that may or may not be useful to you.