How to create one big dependency map for the whole project that is created using make files?

Question

I have a gigantic project that is built using make files. Running make at the root of the project takes more than 20 minutes when the files have not been changed (i.e., just moving the project and checking the files).

I would like to create a dependency map that tells me which directories that I need to run make on the basis of the files (s) have been changed. I already have a list of updated files that I can get from my version control system, and I would like to skip 20 minutes and go straight to the places that need to be recompiled.

The project combines several languages ​​and custom tools, so ideally it does not depend on the language (i.e. process all make files only to create dependencies). I will also agree to a solution based on C / C ++, since most of the project is in C ++. The project is built on Linux.

[Change - clarifications, replies to comments]

The project is truly gigantic, it takes 20 minutes simply because of the size and because of all the things described in the link below ("recursive will be considered harmful"). Unfortunately, the project is made up of many parts that come from different places, and a little flexibility in what I can do with low-level mafia files ... those that combine elements on the upper level are the ones that I manage.

Answer 1

One thing you can find is a "makefile graph" - there are several projects for creating dependency trees from large makefile projects. Here's one: http://sailhome.cs.queensu.ca/~bram/makao/index.html . I think this will create a beautiful graph and actually build things.

If you want to deploy your own Makefile parser and essentially replace make with your own script, my advice would be "don't do". But there is a collection of Perl modules: Makefile :: Parser . He claims to "pass 51% of the GNU Make test suite."

If you just want to take a look at what took so long, you can turn on Make debug output. Its bits are usually useless, for example, the pages and pages "Make a decision on the application of implicit rules", but perhaps there will be some obvious thing that can not happen. If you want to do this, you can look at remake , "GNU Make with Acceptable Tracing and Debugger."

Answer 2

If a simple make without updates takes 20 minutes, this should be a huge project! Given that your makefile configuration is fine, you can try something like ccache .

note that even if you get a list of files that you just need to merge, you still have to run make from the root of the project, because the changes will need to be linked back to the main binaries. Therefore, if ccache is not good enough and you insist on solving this problem yourself, you still have to keep track of which binaries you need to re-link with "ld"