Upload file to R with POST when sending data

Question

I am trying to download a file to get it from the server, I need to send data at the same time. It works fine with curl on the command line:

curl "https://www.ishares.com/us/product-screener-download.dl" --data "productView=ishares&portfolios=239561-239855"

Unfortunately, I do not work with R. I tried with download.file, download.file with libcurl, curl_download and with httr. (download.file with curl or wget does not work since I'm on a window machine.)

What I tried and did not work with curl:

library("curl")
handle <- new_handle()
handle_setopt(handle, customrequest = "POST")
handle_setform(handle, productView="ishares",portfolios="239561-239855")
curl_download("https://www.ishares.com/us/products/etf-product-list", "./data/ishares-us-etf.xls", handle=handle)

What I tried and did not work with httr:

library(httr)
POST("https://www.ishares.com/us/products/etf-product-list", body = list(productView="ishares",portfolios="239561-239855"))

Answer 1

After searching a bit with Fiddler, I found out that I need to send data with postfield fields, and then everything will be fine.

library("curl")
handle <- new_handle()
handle_setopt(handle, customrequest = "POST")
handle_setopt(handle, postfields='productView=ishares&portfolios=239561-239855')
curl_download("https://www.ishares.com/us/product-screener-download.dl", "./data/ishares-us-etf.xls", handle=handle)

Answer 2

, URL encode = "form" httr::POST().

httr @leo:

library(httr)
POST("https://www.ishares.com/us/product-screener-download.dl",
     body = list(productView = "ishares", portfolios = "239561-239855"),
     encode = "form", write_disk("/tmp/ishares-us-etf.xls"))
#> Response [https://www.ishares.com/us/product-screener-download.dl]
#>   Date: 2016-02-08 06:52
#>   Status: 200
#>   Content-Type: application/vnd.ms-excel;charset=UTF-8
#>   Size: 13.6 kB
#> <ON DISK>  /tmp/ishares-us-etf.xls
head(readLines(file_path), 5)
#>   [1] "<?xml version=\"1.0\"?>"
#>   [2] "<Workbook xmlns=\"urn:schemas-microsoft-com:office:spreadsheet\" xmlns:ss=\"urn:schemas-microsoft-com:office:spreadsheet\">"
#>   [3] "<Styles>"                          
#>   [4] "<Style ss:ID=\"Default\">"
#>   [5] "<Alignment Horizontal=\"Left\"/>"

Answer 3

?

URL <- "https://www.ishares.com/us/products/etf-product-list"
values <- list(productView="ishares", portfolios="239561-239855")
POST(URL, body = values)
r <- GET(URL, query = values)
x <- content(r)