Parallelizing PHP processes using Bash Script?

Question

I want to run ~ 10 php processes from a bash script. When one of them ends, I would like the bash script to start another php process and continue indefinitely, always working on ~ 10 php processes.

What is the easiest way to do this?

The launched php file will be the same every time, but the php process will know to retrieve new values ​​from the database in order to process new data each time. The file I need to run and all its classes are already written in php.

Answer 1

It seems to be suitable for super worship . The following configuration ensures that 10 processes always work, and deals with log rotation, which is also convenient. All products, including stderr, will be recorded in /var/log/worker.log. When "autorestart = true", supervisord will replace the child process as soon as it exits.

[program:worker] command=php /path/to/worker.php process_name=%(program_name)s_%(process_num)d stdout_logfile=/var/log/%(program_name)s.log redirect_stderr=true stdout_capture_maxbytes=512MB stdout_logfile_backups=3 numprocs=10 numprocs_start=0 autostart=true autorestart=true 

Once you have a supervisor configuration (usually / etc / supervisord / conf.d), you can use supervisorctl as a convenient way to start and stop a group of processes.

 $ supervisorctl start worker ... $ supervisorctl stop worker ... $ supervisorctl status worker:worker_0 RUNNING pid 8985, uptime 0:09:24 worker:worker_1 RUNNING pid 10157, uptime 0:08:52 ... worker:worker_9 RUNNING pid 12459, uptime 0:08:31 

Answer 2

You can use GNU Parallel by laying out a list of images to control in parallel , as described here .

Answer 3

You can use something like this. Use one file to run 10 (just run it once), and the bottom of each file can restart itself when it ends.

 /** * Asynchronously execute/include a PHP file. Does not record the output of the file anywhere. * * @param string $filename file to execute, relative to calling script (or root?) * @param string $options (optional) arguments to pass to file via the command line */ function asyncInclude($filename, $options = '') { exec("/path/to/php -f {$filename} {$options} >> /dev/null &"); } 

Answer 4

 jcomeau@intrepid :/tmp$ cat test.sh #!/bin/sh set -m # monitor mode task="php-cgi /tmp/sleep.php" function do_task { $task >/dev/null & echo -n spawned $! ' ' >&2 } trap do_task SIGCHLD for i in $(seq 1 10); do do_task done while true; do wait done jcomeau@intrepid :/tmp$ cat sleep.php <?php sleep(3); ?>