Python - ordered headers for HTTP requests

Question

I am currently using the python 2.7 query library and there is no support for ordered headers. I can put the ordered data for the message and get (for example, an ordered dictionary), but there is simply no header support. Not even in python 3

I know the HTTP RFC protocol, indicates that the order of the headers is not significant, but the problem is that the third-party service that I am implementing does not work if the headers are not in order. I know this because I implemented custom header requests in other languages, and it works (like java), and yes, I am 100% sure of this, because I checked on burp and wirehark to make sure that this is the only one difference between requests, But I already have 5,000 lines in python, so migration causes such a painful solution due to such a problem.

The only solution I thought of is to implement the HTTP protocol on top of TCP itself, but this is not a reasonable solution. I cannot have the same code quality as the available solutions, and this is a possible error for my code.

See a simplified code example below:

data=(("param1","something"), ("param2","something_else")) headers={'id': 'some_random_number', 'version':'some_random_number' , 'signature':'some_random_number' , 'Content-Type':'application/x-www-form-urlencoded' , 'charset':'utf-8' , 'Content-Length':str(len(urllib.urlencode(data))) , 'name':'random' , 'User-Agent':'Firefox' , 'Connection':'Keep-Alive' , 'Accept-Encoding':'gzip'} requests.post("myservice.com",headers=headers, data=data) 

The order of the request headers goes like this (not the actual order, just an example to get my point)

 'version':'some_random_number' 'Accept-Encoding':'gzip' 'id': 'some_random_number' 'User-Agent':'Firefox' 'signature':'some_random_number' 'Connection':'Keep-Alive' 'Content-Type':'application/x-www-form-urlencoded' 'charset':'utf-8' 'name':'random' 

This is a problem for me. I do not know what to do at this moment. Any help is greatly appreciated. I tried urllib library without support

Answer 1

Turning OrderedHeaders in a comment, here is a very, very simple OrderedHeaders , which requests can be satisfied:

 class OrderedHeaders(object): def __init__(self, *headers): self.headers = headers def items(self): return iter(self.headers) oh = OrderedHeaders(('Accept-Charset', 'Foo'), ('Bar', 'Foobar')) for k, v in oh.items(): print("%s:%s" % (k, v)) 

Here is a more detailed example that uses topological sorting to determine which headers should appear before other headers. This requires a bit more code, but you can clearly indicate which sorting of headers you need to have once and use the class like any other dict afterwards.

 import sys import toposort class OrderedHeaders(dict): # The precedence of headers is determined once. In this example, # 'Accept-Encoding' must be sorted behind 'User-Agent' # (if defined) and 'version' must be sorted behind both # 'Accept-Encoding' and 'Connection' (if defined). PRECEDENCE = toposort.toposort_flatten({'Accept-Encoding': {'User-Agent'}, 'version': {'Accept-Encoding', 'Connection'}}) def items(self): s = [] for k, v in dict.items(self): try: prec = self.PRECEDENCE.index(k) except ValueError: # no defined sort for this header, so we put it behind # any other sorted header prec = sys.maxsize s.append((prec, k, v)) return ((k, v) for prec, k, v in sorted(s)) # Initialize like a dict headers = OrderedHeaders(name='random', Connection='Keep-Alive') ... # Setting more values headers['Accept-Encoding'] = 'gzip' headers['version'] = '0.1' headers['User-Agent'] = 'Firefox' ... # Headers come out of '.items()' like they should for k, v in headers.items(): print("%s: %s" % (k, v)) 

prints

 Connection: Keep-Alive User-Agent: Firefox Accept-Encoding: gzip version: 0.1 name: random 

because Connection must arrive before version , User-Agent must arrive before Accept-Encoding , Accept-Encoding must arrive before version , and name has no sorting and therefore must be the last.

You can set the values ​​on OrderedHeaders in any order, sorting is done in .items() . However, you can be sure that sound sequencing is always possible: if you make a mistake and define a circular dependency (for example, "version"> "User-agent"> "version"), you will get toposort.CircularDependencyError in the "compile time" field.