How to check if URL exists with Djangos validators?

Question

I want to check django if the url exists, and if necessary, I want to show something on the screen, that is:

if URL_THAT_POINTS_TO_SOME_PDF exists SHOW_SOMETHING 

Answer 1

Change: Please note this is no longer valid for any version of Django above 1.5

I assume that you want to check if the file actually exists, and not if there is only an object (which is a simple if statement)

Firstly, I will recommend always looking at the source code of Django, because you will find great code that you could use :)

I assume you want to do this in a template. There is no built-in template tag to validate the URL, but you can use this URLValidator class inside the template tag to validate it. Simply:

 from django.core.validators import URLValidator from django.core.exceptions import ValidationError validate = URLValidator(verify_exists=True) try: validate('http://www.somelink.com/to/my.pdf') except ValidationError, e: print e 

The URLValidator class will spit out a ValidationError when it cannot open the link. It uses urllib2 to actually open the request, so it doesn’t just use basic regex checking (but also does it).

You can add this to your custom template tag, which you will learn how to create in django documents, and you're done.

Hope this is the beginning for you.

Answer 2

Everything based on the verify_exists parameter on django.core.validators.URLValidator will stop working with Django 1.5 - the documentation doesn’t say anything about it, but the source code shows that using this mechanism in 1.4 (the latest stable version) leads to DeprecationWarning (you see it is completely removed in the development version ):

 if self.verify_exists: import warnings warnings.warn( "The URLField verify_exists argument has intractable security " "and performance issues. Accordingly, it has been deprecated.", DeprecationWarning ) 

There are also some odd quirks with this method due to the fact that it uses a HEAD request to check URLs - it is efficient with bandwidth, but some sites (like Amazon) respond to an error (before HEAD , where the GET equivalent would be fine ), and this leads to false negative results from the validator .

I would also (change a lot in two years) recommend not doing anything with urllib2 in the template - this is a completely wrong part of the request / response cycle that triggers potentially lengthy operations: consider what happens if the URL exists, but the DNS problem causes urllib2 take 10 seconds. Bam! Instantly 10 extra seconds on page load.

I would say that best practice for creating as long as possible tasks, such as asynchronous (and therefore not blocking page loading), uses django-celery ; there’s a basic tutorial that covers using pycurl to check a website, or you can see how Simon Willison implemented celery tasks (slides 32-41) for a similar purpose on Lanyrd.

Answer 3

It is required in addition:

from django.core.exceptions import ValidationError

so that it works for me. Just saying: 0)

Answer 4

Problem

from django.core.validators import URLValidator says that www.google.ro not valid. What is wrong in my point of view. Or at least not enough.

How to solve it?

Key. Look at the source code for models.URLField , you will see that it uses forms.FormField as a validator. What's more than the top URLValidator

Decision

If I want to check a url as http://www.google.com or as www.google.ro , I would do the following:

from django.forms import URLField

 def validate_url(url): url_form_field = URLField() try: url = url_form_field.clean(url) except ValidationError: return False return True 

I found this helpful. Maybe this helps someone else.

Answer 5

 from django.core.validators import URLValidator from django.core.exceptions import ValidationError validate = URLValidator(verify_exists=True) value = request.GET.get('url', None) if value: try: validate(value) except ValidationError, e: print e 

validate(value) fails if the URL does not precede a scheme like http:// . Interestingly, this is by design.

Answer 6

I did not see the answer here. It might help someone else.

 from django import forms f = forms.URLField() try: f.clean(http://example.com) print "valid url" except: print "invalid url" 

Answer 7

See: http://www.agmweb.ca/2009-04-19-django-urlpatterns---its-more-than-just-urls/

In django 1.10 now I use:

 from django.core.urlresolvers import RegexURLResolver, Resolver404 if 'next' in request.GET.keys(): n = request.GET["next"].strip('/') + "/" resolver = RegexURLResolver(r'', urls) try: callback, callback_args, callback_kwargs = resolver.resolve(n) return HttpResponseRedirect(str(request.GET["next"])) except Resolver404: raise PermissionDenied("This page is not available")