R download from aspx to https get site instead of csv

Question

warning: Newbe is here. I would be grateful for some recommendations. I am trying to make an investment to learn how to use R to automate loading.

What I need: To download shale gas well data from this website for all counties and reporting periods: https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Production/ProductionByCounty.aspx (Please note that the agreement can be set at the entrance, and not in large quantities)

I can go to a page that lists all the CSV files that I want to download. Unfortunately, the site has the same address as above. (You can try to choose the county and reporting period and see for yourself)

However, once on this page are listed links that activate the CSV download. For each of them, there is something like this: https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Production/ProductionByCountyExport.aspx?UNCONVENTIONAL_ONLY=false&INC_HOME_USE_WELLS=true&INC_NELLERELODElEnElElElEerDElElEerDElEerDElEEerDElEEerDElEGerDElEnEerDOlEngDeLEnDElDEnDElDEnDerIoDEnDElEoDerDOlEngElEdEnDerOnDeOlEngElE

What I tried:

library(downloader)

download ("https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Production/ProductionByCountyExport.aspx?UNCONVENTIONAL_ONLY=false&INC_HOME_USE_WELLS=true&INC_NON_PRODUCING_WELLS=true&PERIOD=15AUGU&COUNTY=ALLEGHENY",
          destfile="Prod_AUG15_Allegheny.csv")

I followed what another person did: Downloading documents from an aspx webpage to R

Problem: This command saves the site instead of the csv file.

trying URL 'https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Production/ProductionByCountyExport.aspx?UNCONVENTIONAL_ONLY=false&INC_HOME_USE_WELLS=true&INC_NON_PRODUCING_WELLS=true&PERIOD=15AUGU&COUNTY=ALLEGHENY'
Content type 'text/html; charset=utf-8' length 11592 bytes (11 Kb)
opened URL
downloaded 11 Kb

Question: Is this related to my https page instead of http? Any recommendations on how to solve this problem or other important posts for you? (I might find some posts about aspx downloads but nothing useful)

Thank you in advance

Answer 1

@hrbrmstr ! , , RSelenium .

(, , , ):

# Using RSelenium to save file
##Installing the package if needed
install.packages("RSelenium")
##Activating 
library("RSelenium")
checkForServer()
startServer()
#I had to start the server manually!
remDr <- remoteDriver()
remDr
remDr$open()
#open website and accepting conditions
remDr$navigate("https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Welcome/Agreement.aspx")
AgreeButton<-remDr$findElement(using = 'id', value="MainContent_AgreeButton")
AgreeButton$highlightElement()
AgreeButton$clickElement()

remDr$navigate("https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Production/ProductionByCountyExport.aspx?UNCONVENTIONAL_ONLY=false&INC_HOME_USE_WELLS=true&INC_NON_PRODUCING_WELLS=true&PERIOD=15AUGU&COUNTY=ALLEGHENY")

!! csv:-( , " ..." , RSelenium.

, !