I am trying to run a command to download 3000 files in parallel. I am using Cygwin + Windows.
Uploading a single file via WGET to the terminal:
wget --no-check-certificate --content-disposition --load-cookies cookies.txt \ -p https://username:password@website.webpage.com/folder/document/download/1?type=file
allows me to download the file with identifier 1 separately, in the correct format (as long as it --content-dispositionis on the team).
I repeat this REST API call to download the entire folder (3000 files). This works fine, but rather slowly.
FOR /L %i in (0,1,3000) do wget --no-check-certificate --content-disposition --load-cookies cookies.txt \ -p https://username:password@website.webpage.com/folder/document/download/%i?type=file
Now I am trying to run a program in Cygwin in parallel.
seq 3000 | parallel -j 200 wget --no-check-certificate --content-disposition --load-cookies cookies.txt -p https://username:password@website.webpage.com/folder/document/download/{}?type=file
It starts, but the file name and format are lost (instead "index.html", for example, we can get "4@type=file"as the file name).
Is there any way to fix this?