All geek questions in one place

What should I do if I do not want my site to be indexed by search engines?

What tag should you put in HTML so that your pages are not indexed by search engines?

+3
source share
3 answers

Add this to the HTML <head>element of the page you want to not index:

<meta name="robots" content="noindex, nofollow">

To cover the entire site, create a robots.txtroot folder that contains the following lines:

User-agent: *
Disallow: /

See also:

+12
source

Use the robots.txt file to limit indexing: http://www.robotstxt.org/orig.html

+7
source

. , .

noindex HTML, . , Bing Google , ( ). , noindex, .

, noindex (Google, Bing)..

noindex HTML :

<meta name="robots" content="noindex, noodp, noarchive, noimageindex" />

, "". .

Google Bing robots.txt, noindex, . , Google Bing , noindex " , -", , robots.txt, " - , , ." : Google Bing , , , . , Google Bing noindex.

, noindex (Internet Archive, Alexa, Blekko, Baidu)...

, robots.txt. noindex, , .

  • , sitemap.xml Google Bing, ( !).
  • (, , pdf ..), HTTP- x-robots. . !

...

I launched a site with 7M legal documents. Some of them have personal information and may not be in search engines. I have studied this more than any person ever, and it disappoints that the robots.txt myth is so powerful.

+1
source

Source: https://habr.com/ru/post/1758832/

More articles:


All Articles