We implemented a rating system on the site, and then a link to the script. Nevertheless, with the overwhelming majority of ratings on the site being 3/5 and ratings very much even at 1-5, we are beginning to suspect that search robots, etc. Pass. The URLs used are as follows:
http://www.thesite.com/path/to/the/page/rate?uid=abcdefghijk&value=3
When we started, add the following to our robots.txt:
User-agent: *
Disallow: /rate
Is it wrong or googlebot and others just ignore our robots.txt?
source
share