How to generate a list of all dates in a range using tools available in bash?

Question

I want to upload a bunch of files with ISO-8601 names. Is there an easy way to do this with bash + GNU coreutils? (Or some trick for wget / curl to automatically generate a list, but I find this unlikely)

Similar to this question, but not limited to weekdays: How to create a date range without waiting using the tools available in bash? . I assume there is an easier way to do this without this restriction.

Also applies to How to create a date range for random data in bash , but is not limited to one year.

Answer 1

If you have a GNU date , you can use any for loop in any POSIX-compatible shell :

 # with "for" for i in {1..5}; do echo $(date -I -d "2014-06-28 +$i days") done 

or until loop, this time using the advanced test Bash [[ :

 # with "until" d="2014-06-29" until [[ $d > 2014-07-03 ]]; do echo "$d" d=$(date -I -d "$d + 1 day") done 

Note that non-ancient versions of sh will also perform lexicographic comparisons if you change the condition to [ "$d" \> 2014-07-03 ] .

The output from any of these loops:

 2014-06-29 2014-06-30 2014-07-01 2014-07-02 2014-07-03 

For a more portable way to do the same, you can use the Perl script:

 use strict; use warnings; use Time::Piece; use Time::Seconds; use File::Fetch; my ($t, $end) = map { Time::Piece->strptime($_, "%Y-%m-%d") } @ARGV; while ($t <= $end) { my $url = "http://www.example.com/" . $t->strftime("%F") . ".log"; my $ff = File::Fetch->new( uri => $url ); my $where = $ff->fetch( to => '.' ); # download to current directory $t += ONE_DAY; } 

Time :: Piece, Time :: Seconds and File :: Fetch are the main modules. Use it as perl wget.pl 2014-06-29 2014-07-03 .

Answer 2

Using GNU date and bash:

 start=2014-12-29 end=2015-01-03 while ! [[ $start > $end ]]; do echo $start start=$(date -d "$start + 1 day" +%F) done 

 2014-12-29 2014-12-30 2014-12-31 2015-01-01 2015-01-02 2015-01-03 

Answer 3

Here is how I did it:

 d=$(date -I); while wget "http://www.example.com/$d.log"; do d=$(date -I -d "$d - 1 day"); done 

This is an attempt to download all the files from today until we get 404.

Answer 4

I use this convenient function to work with log files in the format yyyymmdd.log.gz:

 function datelist { for dt in $(seq -w $1 $2) ; do date -d $dt +'%Y%m%d' 2>/dev/null ; done ; } 

It accepts dates in yyyymmdd format.