Httr: getting data using POST ()

Question

Disclaimer: so far I have been able to capture data from another source using the httr POST function, let it be known that I am full n00b regarding httr and HTML forms in general.

I would like to bring some data directly to R from a website using httr. My first attempt was to pass a named list to the arg body (as shown in this vignette ). However, I noticed square brackets in the form input names (at least I think they are input arguments to the form). So instead, I tried to pass the body as a string, as I think it should appear in the request body:

url <- 'http://research.stlouisfed.org/fred2/series/TOTALSA/downloaddata' query <- paste('form[native_frequency]=Monthly', 'form[units]=lin', 'form[frequency]=Monthly', 'form[obs_start_date]="1976-01-01"', 'form[obs_end_date]="2014-11-01"', 'form[file_format]=txt' sep = '&') response <- POST(url, body = query) 

In any case, the above code simply returns the source code of the web page, and I cannot figure out how to submit the form correctly, so that it returns the same data as manually by clicking the "Upload data" button.

In Chrome’s Developer / Web Tools, the response header in the Content-Disposition section indicates that there is a text file attachment containing data when I manually click on the "Load data" button on the form. It doesn't seem to be in any of the headers associated with the response object in the code above. Why this file is not returned with a POST request - where is the data file located?

Looks like I'm missing something obvious. Anyone want to help me connect the dots?

Answer 1

Typically, if you intend to use httr , you allow it to create and encode data for you, you simply pass the information through a list of form values. Try

 url<-"http://research.stlouisfed.org/fred2/series/TOTALSA/downloaddata" query <- list('form[native_frequency]'="Monthly", 'form[units]'="lin", 'form[frequency]'="Monthly", 'form[obs_start_date]'="1996-01-01", 'form[obs_end_date]'="2014-11-01", 'form[file_format]'="txt") response <- POST(url, body = query) content(response, "text") 

and the return looks something like this:

 [1] "Title: Total Vehicle Sales\r\nSeries ID: TOTALSA\r\nSource: US. Bureau of Economic Analysis\r\nRelease: Supplemental Estimates, Motor Vehicles\r\nSeasonal Adjustment: Seasonally Adjusted Annual Rate\r\nFrequency: Monthly\r\nUnits: Millions of Units\r\nDate Range: 1996-01-01 to 2014-11- 01\r\nLast Updated: 2014-12-05 7:16 AM CST\r\nNotes: \r\n\r\nDATE VALUE\r\n1996-01-01 14.8\r\n1996-02-01 15.6\r\n1996-03-01 16.0\r\n1996-04-01 15.5\r\n1996-05-01 16.0\r\n1996-06-01 15.3\r\n1996-07-01 15.1\r\n1996-08-01 15.5\r\n1996-09-01 15.5\r\n1996-10-01 15.3\r