What configuration problems or problems can make the Maven Assembly plugin slow?

Question

Our multi-module Maven project, used to build 4-6 minutes. Over the past few months, it has increased to 20+ minutes per assembly. One symptom is that sometimes the build seems to be paused until I press <enter>. However, the Maven build still works fine (6 minutes, without pauses) on our build server.

Build mvn clean package -D<profile>

Some of our plugins include:

• PMD
• Findbugs
• Assembly
• Thrift Compiler
• jspc-maven plugin
• Maven-substitute-plugin

In addition, we have an internal Nexus repository.

Update: build logs

Local layout:

 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Project Parent ............................. SUCCESS [17.703s] [INFO] Dependencies ............................... SUCCESS [0.109s] [INFO] Thrift Service ............................. SUCCESS [1:51.141s] [INFO] Thrift API Client Sample ................... SUCCESS [14.219s] [INFO] Application ................................ SUCCESS [14:07.984s] [INFO] Webapps Parent ............................. SUCCESS [1.250s] [INFO] Webapp A ................................... SUCCESS [27.547s] [INFO] Webapp B.................................... SUCCESS [20.672s] [INFO] Webapp C ................................... SUCCESS [1:14.656s] [INFO] Assembly ................................... SUCCESS [5:47.219s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 24:23.234s [INFO] Finished at: Fri Jan 27 10:47:38 EST 2012 [INFO] Final Memory: 25M/66M [INFO] ------------------------------------------------------------------------ 

Build Server (Team City):

 [02:16:31]: [INFO] ------------------------------------------------------------------------ [02:16:31]: [INFO] Reactor Summary: [02:16:31]: [INFO] [02:16:31]: [INFO] Project Parent ............................. SUCCESS [3.421s] [02:16:31]: [INFO] Dependencies ............................... SUCCESS [3.157s] [02:16:31]: [INFO] Thrift Service ............................. SUCCESS [41.314s] [02:16:31]: [INFO] Thrift API Client Sample ................... SUCCESS [1.220s] [02:16:31]: [INFO] Application ................................ SUCCESS [4:09.617s] [02:16:31]: [INFO] Webapps Parent ............................. SUCCESS [0.323s] [02:16:31]: [INFO] Webapp A ................................... SUCCESS [12.811s] [02:16:31]: [INFO] Webapp B ................................... SUCCESS [16.496s] [02:16:31]: [INFO] Webapp C ................................... SUCCESS [19.011s] [02:16:31]: [INFO] Assembly ................................... SUCCESS [1:45.872s] [02:16:31]: [INFO] ------------------------------------------------------------------------ [02:16:31]: [INFO] BUILD SUCCESS [02:16:31]: [INFO] ------------------------------------------------------------------------ [02:16:31]: [INFO] Total time: 7:33.655s [02:16:31]: [INFO] Finished at: Wed Jan 25 02:16:31 EST 2012 [02:16:31]: [INFO] Final Memory: 42M/317M [02:16:31]: [INFO] ------------------------------------------------------------------------ 

Update 2

Here is an empirical analysis of where my assembly spends most of its time using the timestamp analysis provided by this bash script: https://gist.github.com/993139

It seems to me that I can turn off FindBugs, PMD and Unit Tests for some collections. But I need the final assembly - Assembly. So let me focus on my question - what can make a build plugin slow?

Update 3

As expected, FindBugs, PMD, and unit tests reduce builds by more than 50%

 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Project Parent .................................... SUCCESS [13.969s] [INFO] Dependencies ...................................... SUCCESS [0.094s] [INFO] Thrift Service .................................... SUCCESS [47.125s] [INFO] Thrift API Client Sample .......................... SUCCESS [11.922s] [INFO] Application ....................................... SUCCESS [3:10.922s] [INFO] Webapps parent .................................... SUCCESS [0.468s] [INFO] Webapp A .......................................... SUCCESS [18.157s] [INFO] Webapp B .......................................... SUCCESS [18.437s] [INFO] Webapp C .......................................... SUCCESS [1:00.672s] [INFO] Assembly .......................................... SUCCESS [3:55.969s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 9:58.609s [INFO] Finished at: Mon Feb 06 10:21:01 EST 2012 [INFO] Final Memory: 24M/59M [INFO] ---------------------------------------------------------------------- 

I do not think that the Assembly should take 4 minutes. During the build phase, I see a lot of the following:

 [INFO] --- maven-assembly-plugin:2.2:single (assembly-full) @ assembly --- [INFO] Reading assembly descriptor: C:\projects\my-project\assembly/src/main/assembly/assembly-full.xml [INFO] project-2.9.3-SNAPSHOT/config/ already added, skipping [INFO] project-2.9.3-SNAPSHOT/config/ already added, skipping [INFO] project-2.9.3-SNAPSHOT/ already added, skipping [INFO] project-2.9.3-SNAPSHOT/var/ already added, skipping 

Repeated with every build artifact. Could this somehow become a source of slowness?

Answer 1

The first recommendation is to build locally and see the console. This should help you easily identify the plugins that take the most time.

From my memory, PMD and FindBugs can take a considerable amount of time. The same goes for javadoc for large projects. Other plugins that you talked about, I can not comment.

Of course, Nexus can be a problem if it runs on a slow system, possibly using slow storage.

Ah, just re-read your post: this is what happens on local machines mostly. What does the local machine do? Is he facing a low memory situation? Where pauses occur, may also be indicated.

Answer 2

I ran into the same problem (40 minutes on build for my project). A study of stacktracke showed that building plugins requires a lot of computational dependency time (computation! Not copying!). Commenting out dependencySet should speed things up significantly if the build plugin is slow because of this for your project.

The workaround for maven 3.0.x is to use the dependency goal : copy-dependencies with the appropriate scope before assembling the plugin, and then use fileSet to include these copied dependencies in the assembly. This solves the performance problem due to some disk space. In my case, assembling the assembly now takes several seconds instead of 40 minutes.

There is still no workaround for maven 3.2.x, as it uses this slow way to calculate dependencies in more places and delays freezing for a long time before starting to do anything. This makes maven 3.2.x unsuitable for large projects.