Bundle python script and dependencies in one file

Question

I have several scripts that had their own copy of some functions, so I extracted these functions into a module and asked the scripts to import this function. These scripts must be copied to a bunch of Linux servers and executed. When the scripts worked autonomously, I would just copy the files to the servers and execute "python".

I have a central management server that will copy and run scripts on different servers.

I read about python eggs, but could use some tips on the method to go for. The way I do it today is to copy and just run a python script. Since this works fine, I thought maybe there is a way to link the scripts themselves to the internal module on which they depend, copy the package to the servers and execute them. I don’t understand why I need to install anything using "pip".

Now, what setting would you recommend? Do I have to build eggs on my local computer and does the management server copy the egg file to the servers? I would prefer to copy everything the server needs from the management server, instead of the servers pulling the dependencies themselves, so I don’t have to punch more holes for all the firewalls. Since eggs usually need to stretch addictions, maybe eggs are not the way to go?

Most of my servers run python 2.6, but I have some running python 2.4 and 3.2.

Hi Kenneth

Answer 1

Perhaps you should take a look at the Twitter PEX library, which can create executables from python packages: https://pex.readthedocs.org/en/latest/whatispex.html

.pex files are simply carefully crafted C #! / usr / bin / env python zip files and special __main.py__

Answer 2

Update 2016 : wagon helps create wheel packages with dependencies for offline installation.

---

For simple projects containing all the sources together in one folder and copying them as a whole, good enough. You can use git to direct your code to a central repository and pull it to your server without creating any packages. Fabric and Ansible are two tools that can help you automate the deployment process. (For example, running git pull remotely and deleting all your pyc files).

If you have common dependencies between projects, pip and wheels are modern alternatives to eggs:

You can create a simple package containing all the dependencies you want to install using.

 $ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX) $ pip wheel -r requirements.txt --wheel-dir=$tempdir $ cwd=`pwd` $ (cd "$tempdir"; tar -cjvf "$cwd/bundled.tar.bz2" *) 

Once you have the package, you can remove it using:

 $ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX) $ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2) $ pip install --force-reinstall --ignore-installed --upgrade --no-index --use-wheel --no-deps $tempdir/* 

(From pip docs )