HTTP GET packet sniffer in Scapy

Question

I am trying to encode a simple sniffer in Scapy, which only prints HTTP packets using the GET method only. Here is the code:

#!/usr/bin/python from scapy.all import * def http_header(packet): http_packet=str(packet) if http_packet.find('GET'): print GET_print(packet) print packet def GET_print(packet1): print "***************************************GET PACKET****************************************************" print packet1 print "*****************************************************************************************************" sniff(iface='eth0',prn=http_header) 

Here is the result:

 ***************************************************************************************************** None T   ) pEa  @@   h  #/  t  }LGku   U oTE  I(  Ͻ 9qi   S  ?   XuW F=   - k=X:  ***************************************GET PACKET**************************************************** T   ) pE   @@   h  #/  t ʪLGku     oTE  I K  AH * e  > v1#D (mG5T o ?  8  喷╭   Ի " KT^ ' mB   ]     k>  _x X     8V? Ǽw/ Z =   N À  \r     )+}   l c 9  j;   h  5 T 9Hۖ/O  )  P މY qf爂 % _`  6x  5D I3   O  t  tpI#     $IC  E    G  J  α   = ]  vһ   b5^|P  DK )uq 2  ț w  tB      y=   n i r .D6 kI a   6iC   c'  0dPqED 4    [ [  hGh̃  ~|Y/ >`\6yP Dq١?T  Mѵ   f ;   Җ  Ǵ gY   di _x 8| eo p xW9  =   vŅYe } T ۨɑy ^ C - _( < {    }       r $  J k- 9    } Ϡf 27  QKԛ ` GY 8  Sh    Y@8  E9 Rϔ &a /vk  6 DF` /9 I d(   -  [A   )pP  y\ռj]   8 _   vf b    I7       + P<_` ***************************************************************************************************** 

I am waiting:

 GET / HTTP/1.1 Host: google.com User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140722 Firefox/24.0 Iceweasel/24.7.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Cookie: PREF=ID=758a20b5fbd4eac9:U=2b2dedf6c84b001f:FF=0:TM=1412150291:LM=1415430021:S=Q-QemmrLqsSsEA9i; NID=67=mRdkPVhtImrOTLi5I1e5JM22J7g26jAcdiDEjj9C5q0H5jj0DWRX27hCM7gLJBeiowW-8omSv-1ycH595SW2InWX2n1JMMNh6b6ZrRsZ9zOCC2a-vstOQnBDSJu6K9LO Connection: keep-alive 

What can I do to get the expected result?

Answer 1

You need to use the sprintf function of the package instead of printing the package itself. You also need to split the string returned from it and combine it together with the newline characters, otherwise it spits everything out on one line:

 #!/usr/bin/python from scapy.all import * def http_header(packet): http_packet=str(packet) if http_packet.find('GET'): return GET_print(packet) def GET_print(packet1): ret = "***************************************GET PACKET****************************************************\n" ret += "\n".join(packet1.sprintf("{Raw:%Raw.load%}\n").split(r"\r\n")) ret += "*****************************************************************************************************\n" return ret sniff(iface='eth0', prn=http_header, filter="tcp port 80") 

I also added a filter for TCP port 80, but this can be removed if you need to.

Output Example:

 ***************************************GET PACKET**************************************************** 'GET /projects/scapy/doc/usage.html HTTP/1.1 Host: www.secdev.org Connection: keep-alive Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.65 Safari/537.36 Referer: https://www.google.co.uk/ Accept-Encoding: gzip, deflate, sdch Accept-Language: en-GB,en;q=0.8,en-US;q=0.6 If-None-Match: "28c84-48498d5654df67640-gzip" If-Modified-Since: Mon, 19 Apr 2010 15:44:17 GMT ' ***************************************************************************************************** 

Pierre points out that you can completely eliminate the http_header function by using the lfilter argument to sniff() . I let the code be a little more concise at the same time:

 #!/usr/bin/python from scapy.all import * stars = lambda n: "*" * n def GET_print(packet): return "\n".join(( stars(40) + "GET PACKET" + stars(40), "\n".join(packet.sprintf("{Raw:%Raw.load%}").split(r"\r\n")), stars(90))) sniff( iface='eth0', prn=GET_print, lfilter=lambda p: "GET" in str(p), filter="tcp port 80") 

Answer 2

EDIT:

Note that Scapy-http now deprecated and included in Scapy 2.4. 3+. Use import scapy.layers.http or load_layer("http") to enable it.

The answer is :

~~ There is a scapy http module that you can install by running pip install scapy-http . Once this is installed, you can import it by running import scapy_http.http . This is separate from your scapy module, but adds functionality to scapy, so you still need to import scapy as usual. ~~

After importing, change the filter string to

 sniff(iface="eth0", prn=GET_print, lfilter= lambda x: x.haslayer(scapy_http.http.HTTPRequest)) 

I removed the filter="tcp and port 80" option because using http lfilter will return all HTTP request requests regardless of the port, with the exception of SSL, for the obvious reason that it cannot be intercepted under normal circumstances. You might want to keep the filter option for performance reasons.

Answer 3

I commented on one way to improve it, but decided to put together an increasingly complete solution. This will not break the asterisk packages, but instead simply prints the headers in a pretty printed dictionary so that it may work for you or it may not, but you can also customize it to suit your needs. Besides formatting, this seems to be the most efficient tool posted on this issue so far, and you can delegate functions to add formatting and further deconstruct the dict.

 #!/usr/bin/env python2 import argparse import pprint import sys # Suppress scapy warning if no default route for IPv6. This needs to be done before the import from scapy. import logging logging.getLogger("scapy.runtime").setLevel(logging.ERROR) # Try to import sniff from scapy.all and show error w/ install instructions if it cannot be imported. try: from scapy.all import sniff except ImportError: sys.stderr.write("ERROR: You must have scapy installed.\n") sys.stderr.write("You can install it by running: sudo pip install -U 'scapy>=2.3,<2.4'") exit(1) # Try to import scapy_http.http and show error w/ install instructions if it cannot be imported. try: import scapy_http.http except ImportError: sys.stderr.write("ERROR: You must have scapy-http installed.\n") sys.stderr.write("You can install it by running: sudo pip install -U 'scapy>=1.8'") exit(1) if __name__ == "__main__": # Parser command line arguments and make them available. parser = argparse.ArgumentParser( formatter_class=argparse.ArgumentDefaultsHelpFormatter, description="Print HTTP Request headers (must be run as root or with capabilities to sniff).", ) parser.add_argument("--interface", "-i", help="Which interface to sniff on.", default="eth0") parser.add_argument("--filter", "-f", help='BPF formatted packet filter.', default="tcp and port 80") parser.add_argument("--count", "-c", help="Number of packets to capture. 0 is unlimited.", type=int, default=0) args = parser.parse_args() # Sniff for the data and print it using lambda instead of writing a function to pretty print. # There is no reason not to use a function you write for this but I just wanted to keep the example simply while # demoing how to only match HTTP requests and to access the HTTP headers as pre-created dict instead of # parsing the data as a string. sniff(iface=args.interface, promisc=False, filter=args.filter, lfilter=lambda x: x.haslayer(scapy_http.http.HTTPRequest), prn=lambda pkt: pprint.pprint(pkt.getlayer(scapy_http.http.HTTPRequest).fields, indent=4), count=args.count )