Is there an alternative to parse_qs that handles half-columns?

Question

TL DR

What libraries / calls are available for processing query strings containing half-columns differently than parse_qs?

>>> urlparse.parse_qs("tagged=python;ruby") >>> {'tagged': ['python']} 

Full background

I am working with the StackExchange API to search for flagged questions.

Search is laid out like this, with tags separated by half-columns:

/2.1/search?order=desc&sort=activity&tagged=python;ruby&site=stackoverflow

Interacting with the API is great. The problem occurs when I want to test calls, especially when using httpretty to mock HTTP.

Under the hood, httpretty uses urlparse.parse_qs from the standard python libraries to parse the request.

 >>> urlparse.parse_qs("tagged=python;ruby") {'tagged': ['python']} 

Clearly this does not work well. This is a small example, here is a fragment of httpretty (out of context of testing).

 import requests import httpretty httpretty.enable() httpretty.register_uri(httpretty.GET, "https://api.stackexchange.com/2.1/search", body='{"items":[]}') resp = requests.get("https://api.stackexchange.com/2.1/search", params={"tagged":"python;ruby"}) httpretty_request = httpretty.last_request() print(httpretty_request.querystring) httpretty.disable() httpretty.reset() 

I want to use machinery from httpretty, but a parse_qs is required for parse_qs . At the moment, I can apply the httpretty patch clown, but I would like to see what else can be done.

Answer 1

To get around this, I temporarily httpretty.core.unquote_utf8 (technically httpretty.compat.unquote_utf8 ).

 # # To get around how parse_qs works (urlparse, under the hood of # httpretty), we'll leave the semi colon quoted. # # See https://github.com/gabrielfalcao/HTTPretty/issues/134 orig_unquote = httpretty.core.unquote_utf8 httpretty.core.unquote_utf8 = (lambda x: x) # It should handle tags as a list httpretty.register_uri(httpretty.GET, "https://api.stackexchange.com/2.1/search", body=param_check_callback({'tagged': 'python;dog'})) search_questions(since=since, tags=["python", "dog"], site="pets") ... # Back to normal for the rest httpretty.core.unquote_utf8 = orig_unquote # Test the test by making sure this is back to normal assert httpretty.core.unquote_utf8("%3B") == ";" 

This suggests that you do not need anything extra. Another option is to leave only half parse_qs with a percentage before it reaches parse_qs .