All geek questions in one place

Will googlebot index my site?

in my robots.txt file, I have the following line

User Agent: Googlebot-Mobile Deny: /

User-Agent: GoogleBot Deny: /

Sitemap: http://mydomain.com/sitemapindex.xml

I know that if I put the first 4 lines, googlebot will not index sites, but what if I put the last line of the Sitemap: http://mydomain.com/sitemapindex.xml , can googlebot index the site?

Thanks,

+3
source share
3 answers

robots.txt ( ), Googlebot Googlebot-Mobile , .

, , robots.txt Sitemap.

, - , . sitemap , , .

, , - sitemaps , , , , .

+4

, , Google . . robots.txt, , . "" "". Robots.txt - , .

+2

googlebot sitemapindex.xml

  • robots.txt .
  • sitemap.xml googlebot.
  • googlebot sitemapindex.xml
  • , ,

robots.txt Google - googlebot ( "" ).

+1

Source: https://habr.com/ru/post/1706087/

More articles:


All Articles