How to wait for an array of torque jobs to complete

Question

I have a script that breaks a data structure into pieces. Pieces are processed using an array of torque jobs and then combined back into a single structure.

The merge operation depends on the filling of the task array. How to make the merge operation wait for the completion of the task array job?

$ qsub --version
Version: 4.1.6

My script looks like this:

# Splits the data structure and processes the chunks
qsub -t 1-100 -l nodes=1:ppn=40,walltime=48:00:00,vmem=120G ./job.sh
# Merges the processed chunks back into a single structure
./merge.sh

I tried:

qsub -t 1-100 -l nodes=1:ppn=40,walltime=48:00:00,vmem=120G -N job1 ./job.sh
qsub -W depend=afterokarray:job1 ./merge.sh

and:

qsub -t 1-100 -l nodes=1:ppn=40,walltime=48:00:00,vmem=120G -N job1 ./job.sh
qsub -hold_jid job1 ./merge.sh

None of them worked. The former led to the error [qsub: illegal -W value], and the latter also led to the error: qsub: script file "job1" cannot be loaded - there is no such file or directory.

Answer 1

Conclusion

qsub -t 1-100 -l nodes=1:ppn=40,walltime=48:00:00,vmem=120G -N job1 ./job.sh

. bash:

FIRST=`qsub first_1.sh`
qsub -W depend=afterok:$FIRST second_1.sh

Answer 2

afterokarray:

ID=$(qsub -t 1-100 -l nodes=1:ppn=40,walltime=48:00:00,vmem=120G -N job1 ./job.sh)
qsub -W depend=afterokarray:$(ID) ./merge.sh

, , , :

#!/bin/bash
#PBS -q batch
#PBS -l walltime=24:00:00
#PBS -o /app/run/KLFLO/nueva_tarea_0.$PBS_ARRAYID.out
#PBS -e /app/run/KLFLO/nueva_tarea_0.$PBS_ARRAYID.err
#PBS -N KLFLO_nueva_tarea_0
#PBS -t 1-2 # times 
sleep 20
/bin/cat /etc/hosts

qsub < nueva_tarea_2.bash (10 []. docker)

#!/bin/bash
#PBS -q batch
#PBS -l walltime=24:00:00
#PBS -o /app/run/KLFLO/nueva_tarea_2.$PBS_ARRAYID.out
#PBS -e /app/run/KLFLO/nueva_tarea_2.$PBS_ARRAYID.err
#PBS -N KLFLO_nueva_tarea_2
#PBS -t 1-2 # times
#PBS -W depend=afterokarray:10[].docker
/bin/cat /etc/hosts