How fast does Google crawl a new page, and can we influence Google’s crawler?

Question

I want to submit my site to Google. How long does it take to crawl a new message on a website?

Also, is there a way to send this message to Googlebot right after the message is created?

Answer 1

Google has three ways to enter a website into its results: search, crawl, indexing.

In order to "discover" your site, it must be aware of its existence - usually through backlinks. If you are a new site, you can use the submit URL form , but this is not a really reliable method. You better subscribe to Google Webmaster Tools and submit your site. An additional step is to provide the XML site of your site. If you publish on your website in a blog / publication, you can always consider PubSubHubbub .

From there, the crawl frequency is usually based on the popularity of the site (as measured by the old PageRank). Depth of scan (crawl-budget) is also determined by PR.

Answer 2

There are several ways to help feed the Google Crawler URL.

The first way is to go here and send the URL ---> www.google.com/webmasters/tools/submit-url/

The second way is to go to Google Webmaster Tools and click "Get as GoogleBot"

And then enter the URL you want to add:

http://i.stack.imgur.com/Q3Iva.png

Then the URL will look something like this:

http: \\ example.site Web success URL sent to index 1/22/12 2:51 AM

---

As for how long it takes for a question to appear on google, there are many factors that can be contributed to this.

If the site owners use Google Webmaster Tools, the following setting is available:

http://i.stack.imgur.com/RqvOi.png

Answer 3

For a quick crawl, you need to send the XML sitemap to google webmaster and manually crawl and index the URL of the web pages using Google Webmaster fetch.
I also used the Google and index crawl method, and after that this practice gives me a better result.

Answer 4

I admitted a mistake in my answer by adding a comment to the original question a long time ago. Now I am updating this post in the interest of keeping future readers astray like me. See Notes from other users below - they are correct. Google does not use the revisit-after meta tag. I still keep the source text of the answer here to make sure that anyone else looking for a similar answer will find it here with this entry, confirming that this meta tag is NOT VAL! Hope this helps someone.

You can use the HTML meta tag as follows:

<meta name="revisit-after" content="1 day"> 

Adjust the time period if necessary. There is no guarantee that robots will return to the set time frame, but that is how you tell robots how often this page can change.

• Meta Tag Revisit is used to tell search engines when they will return.