GNU make -j option

Question

Since I learned about -j, I have used -j8 blithely. The other day I was setting up an atlas, and make failed. In the end, I tracked it down to things that were out of order - and it worked fine once I got back to singlethreaded make. It makes me nervous. What conditions do I need to monitor when writing my own makefiles so as not to do something unexpected with make -j?

Answer 1

I think make -j will respect the dependencies you specify in your Makefile; that is, if you indicate that objA depends on objB and objC, then make will not start working with objA until objB and objC are complete.

Most likely, your Makefile does not accurately determine the necessary order of operations, and it’s just fortunate that this works for you in a single-threaded case.

Answer 2

In short - make sure your dependencies are true and complete.

If you use a single-threaded brand, you can blindly ignore implicit dependencies between goals. When using parallel make, you cannot rely on implicit dependencies. All must be clear. This is probably the most common trap. In particular, if you use .phonic goals as dependencies.

This link is a good tutorial on some issues with parallel make.

Answer 3

Here is an example of the problem that I encountered when I started using parallel assemblies. I have a goal called "fresh" that I use to restore the target from scratch (a "new" build). I used to encode a “fresh” target by simply specifying “clean” and then “build” as dependencies.

build: ## builds the default target clean: ## removes generated files fresh: clean build ## works for -j1 but fails for -j2 

This worked fine until I started using parallel assemblies, but with parallel assemblies, it tries to do a clean and an assembly at the same time. Therefore, I changed the definition of "fresh" as follows to guarantee the correct order of operations.

 fresh: $(MAKE) clean $(MAKE) build 

This is basically just a matter of defining dependencies correctly. The trick is that parallel assemblies are more stringent than single-threaded assemblies. My example shows that the list of dependencies for a given target does not necessarily indicate the order of execution.

Answer 4

If you have a recursive make, things can break easily. If you are not doing a recursive make, then as long as your dependencies are correct and complete, you should not run into any problems (except for an error in make). See Recursive analysis is considered harmful for a more detailed description of problems with recursive make.

Answer 5

A good idea is an automated test to test the -j option for all make files. Even the best developers have problems with the -j make option. The most common problems are the simplest.

 myrule: subrule1 subrule2 echo done subrule1: echo hello subrule2: echo world 

In normal make you will see hello -> world -> done. With make -j 4 you can see the world -> hello -> done

Where I see this happens most often with the creation of output directories. For example:

 build: $(DIRS) $(OBJECTS) echo done $(DIRS): -@mkdir -p $@ $(OBJECTS): $(CC) ... 

Answer 6

I just thought that I would add an answer to the subset, since it does not show the effect clearly. However, adding some sleep commands. Well, it works on Linux.

Then doing make shows the differences with:

• to do
• make -j4

---

 all: toprule1 toprule1: botrule2 subrule1 subrule2 @echo toprule 1 start @sleep 0.01 @echo toprule 1 done subrule1: botrule1 @echo subrule 1 start @sleep 0.08 @echo subrule 1 done subrule2: botrule1 @echo subrule 2 start @sleep 0.05 @echo subrule 2 done botrule1: @echo botrule 1 start @sleep 0.20 @echo "botrule 1 done (good prerequiste in sub)" botrule2: @echo "botrule 2 start" @sleep 0.30 @echo "botrule 2 done (bad prerequiste in top)"