Can Webbots remove content from my site using links that are associated with GET requests?

Question

OK here is my problem: the content disappears from my site . This is not the safest site, it has a number of problems. Right now, every time I load a page that can remove content from my site using simple links connected to a GET request, I find that the relevant content is deleted in bulk.

For example, I have a function on my site for uploading images. When a user uploads an image, the administrator (owner) can use another page to delete all (owned) images from the site. The removal functionality is implemented in such a way that the user clicks on the link under each thumbnail of the downloaded images, which he would send a request for receipt, which removes information about the image from the site database and removes the image from the server file system.

The other day I downloaded this functionality, and the next morning I found that all my images were deleted. Pages are protected by user authentication when viewing pages using a browser. However, to my surprise, I was able wget on this page to solve any problem.

So I was wondering if some kind of evil web bot is deleting my content using these links? Is it possible? What do you advise to further consolidate my site.

Answer 1

It is absolutely possible. Not even evil web bots can do this. The Google bot does not know which link it has has any specific functionality.

The easiest way to possibly solve this problem is to install the correct robots.txt file to tell the bots that they don’t get to certain pages. Start here: http://www.robotstxt.org/

Answer 2

RFC 2616 (HTTP), Section 9.1.1: Secure Methods :

It was found that the GET and HEAD methods SHOULD NOT have the value of taking any action other than a search. These methods should be considered "safe." This allows user agents to present other methods, such as POST, PUT, and DELETE, in a special way, so that the user becomes aware that a possible unsafe action is being requested.

Basically, if your application allows deletion via GET requests, this does it wrong. Bots will follow publicly accessible links, and they don’t have to expect what will succeed, nor will browsers do. If the links are protected, it may be a prefetch of the browser or some kind of acceleration.

Edit: It could be Bing. Internet Explorer is currently sending data to Microsoft about how you are going to collect data for your crappy search engine.

Answer 3

Typically, a crawler crawls a page for any links and looks at those links to see which pages are behind it. So yes, if both have access to this page, the page contains links for deleting items / things, and both of them open these links to see what's behind them, the code just runs.

There are several ways to block bots from scanning pages. Take a look at the robot.txt implementation. In addition, you may need to examine the mechanism / security of your administrator authentication system ... ;-)

Answer 4

You can use the robots.txt file to block access for some web bots. And for those who are not looking for a robots.txt file, you can also use javascript, there should not be many web browsers interpreting it.

<a href="#" onclick="document.location.href='./delete'; return false;">delete</a>