Make (install from source) python without running tests

Question

I am compiling python from source tar. Everything works well, but the tests work 2 hours and two times. How to get around these tests?

0:16:20 [178/405] test_inspect 0:16:26 [179/405] test_int 0:16:27 [180/405] test_int_literal 0:16:27 [181/405] test_io 0:18:18 [182/405] test_ioctl -- test_io passed in 1 min 51 sec 0:18:19 [183/405] test_ipaddress 0:18:22 [184/405] test_isinstance 0:18:23 [185/405] test_iter 0:18:24 [186/405] test_iterlen 0:18:25 [187/405] test_itertools 0:19:09 [188/405] test_json -- test_itertools passed in 44 sec 0:19:30 [189/405] test_keyword 

As a result

 make 7724,86s user 188,63s system 101% cpu 2:10:18,93 total 

I do its distribution as follows

 PYTHON_VERSION = 3.6.1 PYTHON_URL = https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz wget -O dl/Python-${PYTHON_VERSION}.tar.xz ${PYTHON_URL} cd dl tar xf Python-${PYTHON_VERSION}.tar.xz mkdir -p dl/Python-${PYTHON_VERSION}-build/ cd Python-${PYTHON_VERSION} ./configure --enable-optimizations --prefix=$$(pwd)-build --cache-file=$$(pwd)/cache-file 

These commands run tests twice:

 make -C dl/Python-${PYTHON_VERSION} -j8 make -C dl/Python-${PYTHON_VERSION} -j8 install 

ps This is part of another make file.

Answer 1

The configure --enable-optimizations option allows you to run test classes to generate data for Python profiling. The resulting python binary has better performance when executing python code. Improvements are noted here.

 From configure help: --enable-optimizations Enable expensive optimizations (PGO, etc). Disabled by default. 

From Wikipedia

  profile-guided optimisation uses the results of profiling test runs of the instrumented program to optimize the final generated code. 

In short, you should not skip tests when using --enable-optimizations, since the data needed for profiling is generated when tests are run. You can run make -j8 build_all and then make -j8 install to skip the tests once (the tests will still be executed using install target), but this will hit the target. Instead, you can clear the configure flag for better build time.

Answer 2

I did a little (quick) research on skipping test runs when creating Python, specifying either:

• configure - passing some arguments (e.g. --without-tests , --disable-tests , --skip-tests )
• make - an indication of some variable (through the environment or command line)

The first did not give any results. The latter (by searching in the Makefile template) showed that the test execution is called by calling $ {PYTHON_SRC_DIR} /Tools/scripts/run_tests.py (which sets up some things and calls another script that calls another ...). Note that I found the file in Python3.5 (.4) and Python3.6 (.4), but not in Python2.7 (.14). A little more research has shown that you can skip the (higher) test run. What you need to do:

 make -C dl/Python-${PYTHON_VERSION} -j8 EXTRATESTOPTS=--list-tests install 

Notes

• Google didn’t show anything (relevant) on EXTRATESTOPTS , so I assume it’s not officially supported
• You can also set EXTRATESTOPTS=--list-tests as an environment variable before running (internal) make
• Needless to say, if during the build some “minor” error occurred (for example, a non-critical external module (for example, _ssl.so, for example) could not be built), the tests will not fail, so you will find out about it at runtime (which it would be terribly unpleasant if this happened in production).

@ EDIT0

After @amohr's comment, I decided to play a little, so I completed the whole process:

• configure (opts)
• make (opts)
• make install

on an Lnx (Ubtu16) machine with two processors, where one (full) test run takes ~ 24 minutes. Here are my findings (Python3.6):

• It successfully passed in Python3.5 (.4)
• The solution I proposed earlier works in step 3 ^rd, so it skips the test run 2 ^nd : it runs on the (root) Makefile test target ( make test ), which is called install

• As for the 1 ^st test run, checking the Makefile and concluding, this is what I found out, what happens with 2 ^nd ( make ):

  • C sources are built "normally"
  • Tests are running (I read that some profile data is stored somewhere)
  • C sources are rebuilt with different flags (for example, in my case gcc -fprofile-generate was replaced with -fprofile-use -fprofile-correction - check [GNU]: parameters that Optimize control for more information) to use profile information, generated at the previous (auxiliary) step.

• Skipping a test run of 1 ^st will automatically imply without optimization . Way of achievement:

  • make build_all (at 2 ^nd ) - as suggested by other answers

    • Here is the snippet of the (root) Makefile generated by the configure option ( with --enable-optimizations ):

       all: profile-opt build_all: check-clean-src $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks \ Programs/_testembed python-config 

    • And here is one without :

       all: build_all build_all: check-clean-src $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks \ Programs/_testembed python-config 

    • As you can see, the work:

      • configure --enable-optimizations
      • make build_all

      identical to:

      • configure
      • make
  • Manually modifies the (root) Makefile between 1 ^st ( configure --enable-optimizations ) and 2 ^nr ( make ) steps:
    • Find the macro definition PROFILE_TASK=-m test.regrtest --pgo (for me it was around line ~ 250)
    • Add --list-tests to the end
    • Steps ((2.) 1 and (2.) 3.) exactly coincide, but for (2.) 2. tests are not performed. This may mean that:
      • 2 ^nd build sources are identical to 1 ^st (which would make it completely useless)
      • 2 ^nd does some optimization (without any information), which means that it may get corrupted at runtime (I think / hope this is the first case)

Answer 3

The default build goal for optimized builds involves running tests. to skip them try:

  make -C dl/Python-${PYTHON_VERSION} -j8 build_all 

Answer 4

just assemble and install with

 make -j8 build_all make -j8 altinstall