What is the best way to get RSS, in real time or nearly

Question

I would like to know what is the best way to get RSS feeds in real time without downloading the entire feed, even if it has not been modified. I'm really not against the language, I'm just looking for the best way to do this.

Answer 1

You can use the options header header header ETagand If-Modified-Since.

Here is an example python code:

etag = ... # etag of previous request
last_modifier = ... # time of last request

req = urllib2.Request(url)
if etag:
    req.add_header("If-None-Match", etag)

if last_modified:
    req.add_header("If-Modified-Since", last_modified)

opener = urllib2.build_opener(NotModifiedHandler())
url_handle = opener.open(req)
headers = url_handle.info()

if hasattr(url_handle, 'code') and url_handle.code == 304:
    # no change happened
else:
    # RSS Feed has changed

The code can be ported to any language in which you simply add the necessary header tags and check the returned code.

UPDATE: Checkout this blog post: HTTP Conditional GET for RSS Hackers