Robots.txt and Google Calendar

I am looking for the best solution on how I can make sure that I am doing this correctly:

I have a calendar on my website where users can accept the iCal calendar channel and import it into external calendars of their choice (Outlook, iCal, Google Calendar, etc.).

To prevent bad people from crawling / searching my site for * .ics files, I installed Robots.txt to prevent the folders where the feeds are stored.

So essentially the iCal feed might look like this: webcal: //www.mysite.com/feeds/cal/a9d90309dafda390d09/feed.ics

I understand that the above is still publicly available. However, I have a function in which the user can change the address of his channel if he wants.

My question is: in all external calendars there are no problems with importing / subscribing to the calendar channel, with the exception of Google Calendar. It displays a message: Google was unable to crawl the URL due to the robots.txt restriction . Google answer to this question .

Therefore, after searching, I found that the following works:

1) Set up a PHP file (which I use), which essentially forces the file to load. It basically looks like this:

<?php
$url = "/home/path/to/local/feed/".$_GET['url'];
 $file = fopen ($url, "r");
 if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
  }
 while (!feof ($file)) {
  $line = fgets ($file, 1024);
 print $line;
}
fclose($file);
?>

I tried using this script and it seemed to work with Google Calendar without any problems. (Although, I'm not sure if it is updating / updating yet. I'm still waiting to make sure this works).

:: ? Robots.txt, *.ics .

+3
2

, :

  • -.
  • , robots.txt, Googlebot .

robots.txt. , robots.txt . badbots , - / IP-.

robots.txt . http://facebook.com/robots.txt. , Google ( Adsense, , , ), , Google Google .

+1

, robots.txt .

User-agent: Googlebot
Allow: /*.ics$
Disallow: /

User-agent: *
Disallow: /

.ics, ( ). .

, allow , , , . robots.txt - Google http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449

+4

Source: https://habr.com/ru/post/1785331/


All Articles