The most effective way to test links

Question

I am currently developing an application that scans all files on the server and checks each hrefs to see if they are valid or not. Using WebClient or HttpWebRequest / HttpWebResponse is unnecessarily overworking the process, because it loads the whole page every time, which is useless, I only need to check if the 404 link returns.

What would be the most efficient way? Socket seems like a good way to do this, however I'm not quite sure how this works.

Thank you for sharing your experience!

Answer 1

HEAD .

var request = WebRequest.Create("http://google.com/");
request.Method = "HEAD";
using (var response = (HttpWebResponse)request.GetResponse())
{
    if (response.StatusCode == HttpStatusCode.OK)
    {
        // 200 OK
    }
}

Answer 2

HttpWebRequest.Method "Head". .