Nginx $ request_body notation as hexadecimal

Question

I am trying to write the body of request requests to api, and nginx turns all quotes (and some other characters, such as spaces and tabs) into hexadecimal characters.

Here is my magazine format

log_format postdata '{"ts": "$time_iso8601", "status": $status, "req": "$uri", "meth": "$request_method", "body": "$request_body"}';

Here is what is logged

{"ts": "2015-05-20T15:31:11-07:00", "status": 400, "req": "/v2/track", "meth": "POST", "body": {\x22id\x22:\x22user id\x22}}

How can I prevent this so that the log summary line

{"ts": "2015-05-20T15:31:11-07:00", "status": 400, "req": "/v2/track", "meth": "POST", "body": {"id":"user id"}}

Answer 1

You cannot stop to elude him, and you will have to process it.

Python2 example:

line = '{\x22id\x22:\x22user id\x22}'
line.decode('unicode_escape')
>> u'{"id":"user id"}'

Python3 example:

line = '{\x22id\x22:\x22user id\x22}'
bytes(line, 'utf-8').decode('unicode_escape')
>> '{"id":"user id"}'

Ruby example (from fooobar.com/questions/225106 / ... ):

require 'yaml'
line = '{\x22id\x22:\x22user id\x22}'
YAML.load(%Q(---\n"#{line}"\n))
=> "{\"id\":\"user id\"}"

Note. This last example is useful if, after processing the file with logstash

Answer 2

, -. json . http-

http {
 log_format postdata escape=json $request_body;
 access_log /var/log/nginx/access.log postdata;
 .....
}

Answer 3

Starting with version 1.13, there is an escape = none parameter that disables data shielding.

http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

log_format  api_request_log escape=none '[$time_local] $request \n$request_body';

Answer 4

Like others, there is no way to fix this in the nginx configuration. But it is not difficult to process it. If the request body is JSON-formatted, you are likely to come across a lot of \ x0A (new line) and \ x22 (").

Just clean them using sedbefore looking into the log file.

Here is a command for you: LANG='' sed -E 's/(\\x0A|\\x22)//g' access.log