CMake: in what order are the files processed (Cache, Toolchain, ...)?

This seems like a trivial question, since CMake is a script language, the general answer is strictly sequential. But I came across several cases when it was important when or in what order CMake analyzes certain files. Therefore, I am interested in:

  • Is there documentation available that describes the order in which files (including internal CMake files) are analyzed?
  • Whether the order of the files depends on the version of CMake or some parameters / settings / environment of CMake, including. selected generator or host environment?

Cases to which I have reached so far, where the information above is important:

  • The toolchain file is parsed before the compiler is identified, so you have to populate some CMake variables in the cache first / in the tool binding file: cross-compiling cmake with a specific linker doesn't pass arguments to armlink
  • The toolchain file is analyzed several times, therefore, for example, repeatedly printing messages from the toolchain file: toolchain cmake includes several files
  • The clock with variable visibility outside your main CMakeLists.txt file has been CMakeLists.txt : Executing a command or macro in CMake as the last step before completing the Configure step

Perhaps you know even more.

To find the answer, I tried the following: I have a simple basic CMakeList.txt, as shown below, and run cmake --trace … to analyze the parsing order.

 cmake_minimum_required(VERSION 2.8) include(BeforeProjectCmd.cmake) project(ParserTest CXX) add_subdirectory(LibTarget1) add_subdirectory(LibTarget2) add_executable(ExeTarget Test.cpp) variable_watch(CMAKE_BACKWARDS_COMPATIBILITY) 

When I run, for example, cmake --debug-output --trace -G"Visual Studio 12 2013" -DCMAKE_TOOLCHAIN_FILE:FILE_PATH=Toolchain.txt I got a long trace that I tried to generalize:

 # begin try to read CMakeCache.txt ${CMAKE_BINARY_DIR}/CMakeCache.txt PreLoad.cmake ${CMAKE_BINARY_DIR}/PreLoad.cmake # end try to read β”Œ CMakeLists.txt(1): cmake_minimum_required(VERSION 2.8 ) β”‚ CMakeLists.txt(3): include(BeforeProjectCmd.cmake ) β”‚ β”œβ”€ BeforeProjectCmd.cmake β”‚ β”‚ CMakeLists.txt(5): project(ParserTest CXX ) β”œβ”¬ share/cmake-3.2/Modules/CMakeDetermineSystem.cmake β”‚β”‚ │└─ Toolchain.txt β”‚ β”œβ”¬ ${CMAKE_PLATFORM_INFO_DIR}/CMakeSystem.cmake β”‚β”‚ │└─ Toolchain.txt β”‚ β”œβ”€ share/cmake-3.2/Modules/CMakeSystemSpecificInitialize.cmake β”œβ”¬ share/cmake-3.2/Modules/CMakeDetermineCXXCompiler.cmake β”‚β”œβ”¬ share/cmake-3.2/Modules/CMakeDetermineCompiler.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/Platform/Windows-CXX.cmake … β”‚β”‚β”œ share/cmake-3.2/Modules/CMakeDetermineCompilerId.cmake β”‚β”‚β”œβ”€ share/cmake-3.2/Modules/CMakeCompilerIdDetection.cmake … β”‚β”‚β”œ share/cmake-3.2/Modules/Compiler/MSVC-DetermineCompiler.cmake … β”‚β”œ ${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/3.2.2/CMakeCXXCompiler.cmake β”‚β”œ share/cmake-3.2/Modules/CMakeSystemSpecificInformation.cmake β”‚β”œβ”¬ share/cmake-3.2/Modules/CMakeGenericSystem.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/Platform/Windows.cmake ││└─ share/cmake-3.2/Modules/Platform/WindowsPaths.cmake β”‚β”œ share/cmake-3.2/Modules/CMakeCXXInformation.cmake β”‚β”œβ”¬ share/cmake-3.2/Modules/Compiler/MSVC-CXX.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/Platform/Windows-MSVC-CXX.cmake β”‚β”‚β”œβ”¬ share/cmake-3.2/Modules/Platform/Windows-MSVC.cmake │││└─ share/cmake-3.2/Modules/CMakeRCInformation.cmake β”‚β”‚β”” share/cmake-3.2/Modules/CMakeCommonLanguageInclude.cmake β”‚β”œ share/cmake-3.2/Modules/CMakeTestCXXCompiler.cmake β”‚β”œβ”¬ share/cmake-3.2/Modules/CMakeTestCompilerCommon.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/CMakeDetermineCompilerABI.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/CMakeDetermineCompileFeatures.cmake β”‚β”‚β”œ share/cmake-3.2/Modules/Internal/FeatureTesting.cmake β”‚β”‚β”” share/cmake-3.2/Modules/Compiler/MSVC-CXX-FeatureTests.cmake β”‚β”” ${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/3.2.2/CMakeCXXCompiler.cmake β”‚ β”‚ CMakeLists.txt(7): add_subdirectory(LibTarget1 ) β”‚ β”œβ”€ LibTarget1/CMakeLists.txt β”‚ β”‚ CMakeLists.txt(8): add_subdirectory(LibTarget2 ) β”‚ β”œβ”€ LibTarget2/CMakeLists.txt β”‚ β”‚ CMakeLists.txt(10): add_executable(ExeTarget Test.cpp ) β”‚ CMakeLists.txt(12): variable_watch(CMAKE_BACKWARDS_COMPATIBILITY ) β”‚ β”‚ CMake Debug Log in CMakeLists.txt: β”‚ Variable "CMAKE_BACKWARDS_COMPATIBILITY" was accessed using UNKNOWN_READ_ACCESS with value "". -- Configuring done -- Generating ${CMAKE_BINARY_DIR} -- Generating ${CMAKE_BINARY_DIR}/LibTarget1 -- Generating ${CMAKE_BINARY_DIR}/LibTarget2 -- Generating done # writes ${CMAKE_BINARY_DIR}/CMakeCache.txt 

So, having seen the above conclusion, I have come - so far - to the following conclusion (which, I hope, is true and somewhat general):

  • The CMakeCache.txt file is read only once when the setup is started and written after the completion of the Generation. It simply saves the cache state of the "global variables".
  • The project() command launches most of the CMake detection magic (including reading from the Toolchain.txt file).
  • The toolchain file is read twice. Once before the make / compilation system is detected and once inside the then created CMakeSystem.cmake .
  • The variable_watch() hook can be triggered at any time, so the area in which the "command to execute" is called is called undefined.
+15
cmake
May 28 '15 at 10:15
source share
1 answer

There is no official documentation about this specific internal work of CMake, so please find below a summary of what I have learned about CMake so far ...

Which files are processed depends on

  • Host and Target Operating System
  • Target compiler
  • Your computer environment (variables, registry, installed software)
  • Your project CMake script files that may include
    • Your toolchain file
    • Selected Programming Languages
    • Any external projects / libraries / files / scripts

There are many possible combinations of these parameters, but most of the time CMake does everything possible to automatically determine the correct settings for you, and you do not need to worry about how to do this. The good news is - when you need to know - this follows some internal patterns.

Interestingly, it only slightly depends on the CMake generator you choose.

Initial Step: Detecting and Verifying the Compiler

This basically starts with the project() command. As an example, for example, for the CXX language, the main files for detecting the compiler (see also root files in the output of the question):

  • share/cmake-xy/Modules/CMakeDetermineCXXCompiler.cmake

    This basically tries to determine the compiler's executable location and calls it to get a more specific compiler identifier.

    In addition, it, for example, determines the extensions of the source / output file based on the host computer environment and the target operating system.

  • share/cmake-xy/Modules/CMakeCXXCompiler.cmake.in

    This is the template for storing the result of compiler detection in ${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/xyz/CMakeCXXCompiler.cmake .

    These variables are CMAKE_CXX_COMPILER , CMAKE_CXX_SOURCE_FILE_EXTENSIONS , CMAKE_CXX_IGNORE_EXTENSIONS and CMAKE_CXX_COMPILER_ENV_VAR

  • share/cmake-xy/Modules/CMakeCXXInformation.cmake

    This file sets the base flags for the compiler. This is also when the compiler, host, and target have the greatest impact on tuning with these calls:

     include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_CXX_COMPILER_ID}-CXX-${CMAKE_SYSTEM_PROCESSOR} OPTIONAL) include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_CXX_COMPILER_ID}-CXX OPTIONAL) include(Platform/${CMAKE_SYSTEM_NAME}-${CMAKE_BASE_NAME} OPTIONAL) include(Platform/${CMAKE_SYSTEM_NAME} OPTIONAL) 
  • share/cmake-xy/Modules/CMakeTestCXXCompiler.cmake

    This checks everything, etc. define compiler functions by actually invoking the compiler in simple generated CMake projects.

The results of these steps are stored in cached variables, and these files are special, in which case they are protected by variables such as CMAKE_CXX_COMPILER_LOADED , CMAKE_CXX_INFORMATION_LOADED or CMAKE_CXX_COMPILER_WORKS , so as not to run with each subsequent CMake configuration step again.

Project configuration files: change the default values

There are several ways to change CMake defaults without touching the CMakeLists.txt project CMakeLists.txt .

  • -C <initial-cache> command line option

    This can be used if you want to specify some predefined values ​​(usually you provide using the -D ... option) across several projects over and over again. Like some ways to search for libraries on your computer or some presets used by your company.

  • CMakeCache.txt via e.g. cmake-gui

    cmake-gui allows you to manually change the parameters of your project (edit all non-internal variables in CMakeCache.txt ) before you finally create the build environment.

  • CMAKE_TOOLCHAIN_FILE

    It is mainly used for cross-compiling , but it can more broadly be described as setpoints for the compiler toolkit used.

  • PreLoad.cmake

    More or less the same as the "initial cache" option (see above), but it is not provided through the command line parameter. It should only be in the same directory as your CMakeLists.txt project.

    Note. It supports all CMake script commands, such as if() , but PreLoad.cmake has

    • own range of variables (all that are not cached here do not appear in your main CMakeLists.txt )
    • restrictions that are already known (it works to the rest, so basically you can check CMAKE_GENERATOR )
  • CMAKE_USER_MAKE_RULES_OVERRIDE , CMAKE_USER_MAKE_RULES_OVERRIDE_<LANG>

    This allows you to change the immutable default values ​​after automatic detection using CMake.

    Example : Extending valid CXX source file extensions with .c files

    MakeRulesOverwrite.cmake

     list(APPEND CMAKE_CXX_SOURCE_FILE_EXTENSIONS c) 

    Then you can invoke cmake with something like

     > cmake -D CMAKE_USER_MAKE_RULES_OVERRIDE:PATH=..\MakeRulesOverwrite.cmake .. 
  • CMAKE_PROJECT_ParserTest_INCLUDE

    This is intended to "introduce custom code into the project assembly without changing its source" immediately after your project() command has been processed (and the assembly environment has been detected).

Toolchain.cmake: parsed several times

A toolchain file is read several times when defining a system, compiler, etc.

It's important to know:

  • It is read with every call to try_compile() . And since the try compile needs to create a valid executable, you might need - if you, for example, cross-compiling - before

  • If you modify the tool binding file, CMake will rerun compiler detection (as in the previous line). Which deeply helps to play with your compiler settings.

CMake readaptations: everything comes from the cache

And last, but not least, it is important to know that the above trace shows only the initial step. All consecutive project configurations will occupy almost everything, starting with cached variables and, therefore, will read much fewer files during reconfiguration.

References

+10
Dec 21 '16 at 21:32
source share



All Articles