I am trying to use a perl program in a Windows computing cluster of HTCondor. How HTCondor works on windows, it copies all the dependencies to a temporary directory (used as sorting chroot), and then deletes the directory after the specified output is moved to the specified location.
If I take only perl.exe and perl514.dll and do the task as follows: perl -e "print qq/hello\n/"and tell the clan to run it 200 times, then each replication completes for about 15 seconds, which is an acceptable overhead. This almost all the time spent repeatedly copying files over the network and then deleting them. echo_hello.batrunning 200 times takes more than two seconds per replication.
The problem is that when I try to use my completely redesigned perl distribution of 55 MB and 2,289 files, one “hi” rep takes about 4 minutes to copy and delete, which is unacceptable. When I try to make many starts, the disks on the machines are grind, trying to simultaneously process all file operations in all repetitions, so it does not work at all. I don’t know how long this can end, because I gave up in half an hour and no jobs ended.
I realized that PAR :: Packer can solve the problem, but no. I tried to create print_hello.exeas follows: pp -o print_hello.exe -e "print qq/hello\n/". This is still silent, apparently soaking the file system. I think that the executable PAR :: Packer creates a ton of temporary files, since it extracts the files it needs from the archive. I think the Windows file system is completely choking when there are a bunch of parallel operations with small files.
So, how can I shorten the Perl shorthand that I built for something like 6MB and a dozen files? I really use only a tiny amount of basic modules and don’t need most of the crap in binand lib, but I don’t know how to continue disassembling things in a smart way.
Is there an automated way to remove unnecessary files and modules?
, TCL , " " . Perl, PAR? , PAR , , . ( .)
perl . . my_simulation.exe, per514.dll, . - HTCondor, . , perl, Windows HTCondor, .