Script to download Google web history

How to write a script to download one google web story?

I know about

https://www.google.com/history/

https://www.google.com/history/lookup?hl=en&authuser=0&max=1326122791634447

Meals: https://www.google.com/history/lookup month = 1 & day = 9 & year = 2011 & exit = RSS

but they fail when called programmatically, and not through a browser.

+4
source share
4 answers

I wrote a blog post on how to download the entire history of Google Web using the script I put together.

Everything works directly in your client-side web browser (i.e. data is not transmitted to third-party), and you can load it into a CSV file. You can see the source code here:

http://geeklad.com/tools/google-history/google-history.js

My blog post has a bookmarklet that you can use to easily run a script. It works by accessing a single channel, but iterates through reading the entire history of 1000 records at a time, turning it into a CSV string and making the data downloaded with the click of a button.

I ran it in my own history and successfully uploaded over 130 thousand records that came out to about 30 MB when exported to CSV.

EDIT . It seems that the number of foxes that used my script ran into problems, probably due to some weirdness in their historical data. Unfortunately, since the script does everything in the browser, I cannot debug it when it encounters stories that violate it. If you are a JavaScript developer, use my script, and it looks like your story caused it to break; please feel free to help me fix it and send me any code updates.

+8
source

I tried the GeekLad system, unfortunately, two hacked changes occurred # 1 changed the URL (I changed and posted my own copy, which led to the argument # 2 type = rss no longer working.

I only need timestamps ... so the best / worst hack that I wrote after a while began.

Step 1 - fooobar.com/questions/4914 / ... - Using chrome disables ALL security protocols.

Step 2 - https://gist.github.com/devdave/22b578d562a0dc1a8303

Using contentscript.js and manifest.json, create a chrome extension, host ransack.js locally for any service you want (PHP, Ruby, Python, etc.). Goto https://history.google.com/history/ after installing content extensions in developer mode (unpacked). It will automatically add ransack.js + jQuery to dom, collect the data, and then move on to the next "Later" link.

Every 60 seconds, Google will force you to re-enter the system accidentally, so it does not start and leave, but it works, and if they support cancellation, you can always resort to the Ajax call chain and send the page back to the server for further processing. At full tilt, my nasty script collected 1 page of a second of data.

For moral reasons, I will not help anyone modify this script to get search terms and results, since this process is not authorized by Google (although it is not blocked, apparently), and recommend it only to sufficiently motivated individuals to make them work for them According to my estimates, it took me 3-4 hours to get all 9-year data (90 thousand records) per page every 900 ms or faster.

As long as this thing goes, DO NOT browse the rest of the Internet because Chrome works without any warranty, most of them exist for some reason.

+1
source

You can download your search logs directly from Google (in case downloading using a script is not the main goal),

Steps:

1) Log in and go to https://history.google.com/history/

2) Under the picture of your profile, on the right, you can find the settings icon. See Second Option β€œDownload”. Click on it.

3) Then click "Create Archive", then Google will send you a message in a few minutes.

+1
source

it is possible, before issuing a request to receive a script shuld feed, add the HTTP header of the User-Agent known browser so that Google decides that the request came from this browser.

0
source

Source: https://habr.com/ru/post/1390135/


All Articles