CMake adds unnecessary dependencies between .o and .a files

Question

I have a project managed by CMake with several libraries that have time dependencies of links between them, but each of the libraries can be compiled independently of each other. How to express this CMake so that I can create all the libraries at the same time?

For example, I tried this CMakeLists.txt file:

cmake_minimum_required(VERSION 2.6) project (test) add_library(lib1 STATIC lib1.cpp) add_library(lib2 STATIC lib2.cpp) add_library(lib3 STATIC lib3.cpp) add_executable(main main.cpp) target_link_libraries(lib2 lib1) target_link_libraries(lib3 lib2) target_link_libraries(main lib3) 

Each file simply defines another empty function, for example:

 void f1() {} 

When I type cmake . && make -j4 cmake . && make -j4 , I see this:

 [ 25%] Building CXX object CMakeFiles/lib1.dir/lib1.cpp.o Linking CXX static library liblib1.a [ 25%] Built target lib1 [ 50%] Building CXX object CMakeFiles/lib2.dir/lib2.cpp.o Linking CXX static library liblib2.a [ 50%] Built target lib2 [ 75%] Building CXX object CMakeFiles/lib3.dir/lib3.cpp.o Linking CXX static library liblib3.a [ 75%] Built target lib3 [100%] Building CXX object CMakeFiles/main.dir/main.cpp.o Linking CXX executable main [100%] Built target main 

Despite the fact that I specified -j4 and the compilation of each .cpp file should never depend on any .a files, it expects the previous compiler and link to finish, the next will start. I would rather see something like:

 Building CXX object CMakeFiles/lib1.dir/lib1.cpp.o Building CXX object CMakeFiles/lib2.dir/lib2.cpp.o Building CXX object CMakeFiles/lib3.dir/lib3.cpp.o Building CXX object CMakeFiles/main.dir/main.cpp.o Linking CXX static library liblib1.a Built target lib1 Linking CXX static library liblib2.a Built target lib2 Linking CXX static library liblib3.a Built target lib3 Linking CXX executable main Built target main 

Is it possible to tell CMake that it can simultaneously create all .o files?

In fact, I am doing this in a project with a millionth line, which has about 20 processor cores (with distcc), so this is a huge bottleneck in my build times.

Answer 1

Sequential execution is probably due to link dependencies established between the lib1 , lib2 and lib3 static libraries.

One workflow is to get rid of these static library link dependencies. Since you create static libraries anyway, removing the dependencies will not prevent them from connecting successfully. The main executable file must depend on all libraries:

 cmake_minimum_required(VERSION 2.6) project (test) add_library(lib1 STATIC lib1.cpp) add_library(lib2 STATIC lib2.cpp) add_library(lib3 STATIC lib3.cpp) add_executable(main main.cpp) target_link_libraries(main lib1 lib2 lib3) 

Organized in this way, make -j builds libraries in parallel.

If getting rid of link dependencies is not an option, you can apply the principle "Any problem in computer science can be solved using another layer of indirection" :

 cmake_minimum_required(VERSION 2.6) project (test) add_library(lib1_objects STATIC lib1.cpp) add_library(lib2_objects STATIC lib2.cpp) add_library(lib3_objects STATIC lib3.cpp) add_executable(main main.cpp) add_library(lib1 STATIC empty.cpp) add_library(lib2 STATIC empty.cpp) add_library(lib3 STATIC empty.cpp) target_link_libraries(lib1 lib1_objects) target_link_libraries(lib2 lib2_objects lib1) target_link_libraries(lib3 lib3_objects lib2) target_link_libraries(main lib3) 

This installs auxiliary libraries (for example, lib1_objects ) that have no dependencies and can be built in parallel. The original libraries link to these auxiliary libraries and also establish the necessary link dependencies. empty.cpp is just an empty CPP file source file.