Curl: sleep / delay between requests

Question

I am trying to load exception logs using the following command.

curl --cookie ./flurry.jar -k -L "https://dev.flurry.com/exceptionLogsCsv.do?projectID=49999&versionCut=versionsAll&intervalCut=allTime&direction=1&offset=[0-100:10]" --output "exception#1.csv" 

It works fine and loads csv files based on the offset (10,20,30, etc.). I would like to insert a delay between each request. Is it possible to do this in CURL?

Answer 1

Using bash shell (Linux):

 while : do curl --cookie ./flurry.jar -k -L "https://dev.flurry.com/exceptionLogsCsv.do?projectID=49999&versionCut=versionsAll&intervalCut=allTime&direction=1&offset=[0-100:10]" --output "exception#1.csv" sleep 5m done 

This is an infinite loop, and the delay is set by the sleep command.

Edit On a Windows computer, you can do this trick:

 for /L %i in (0,0,0) do ( curl --cookie ./flurry.jar -k -L "https://dev.flurry.com/exceptionLogsCsv.do?projectID=49999&versionCut=versionsAll&intervalCut=allTime&direction=1&offset=[0-100:10]" --output "exception#1.csv" ping -n XX 127.0.0.1>NUL ) 

The sleep command is not available on Windows. But you can use ping to "emulate" it. Just replace the XX above with the number of seconds you want to delay.

Answer 2

wget has delay options

 wget --wait=seconds 

as well as a random delay option

 wget --random-wait 

Answer 3

in bash, this will stop a random number of seconds in the range of 0-60:

 for d in {0..100..10} do i=`printf "%03d" $d` curl --cookie ./flurry.jar -k -L 'https://dev.flurry.com/exceptionLogsCsv.do?projectID=49999&versionCut=versionsAll&intervalCut=allTime&direction=1&offset='$d --output 'exception'$i'.csv' sleep $(($RANDOM*60/32767)) done