FiddlerCore decrypts sdch response

I get a strange answer from the site I was looking for analysis using FiddlerCore. In chrome tools for developers, if I check the answer, it looks completely normal, not in the violin. The code for the fragment is as follows (which worked fine)

String html = oSession.GetResponseBodyAsString(); 

Returns the following, which is not html, note that this is a sample, not a complete huge string.

 JRHwJNeR\0   \0\0\u0001  D\0 2 \b\0 \u0016 7]<!DOCTYPE html>\n win\"> 

It is also overwhelmed with "\ n" and html, like this

 \n\n\n\n\n \n <meta name=\"treeID\" content=\"dwedxE+pgRQAWIHiFSsAAA==\">\n 

The response headers are as follows:

 Cache-Control:no-cache, no-store Connection:keep-alive Content-Encoding:sdch, gzip Content-Language:en-US Content-Type:text/html;charset=UTF-8 Date:Fri, 28 Oct 2016 10:17:02 GMT Expires:Thu, 01 Jan 1970 00:00:00 GMT Pragma:no-cache Server:Apache-Coyote/1.1 Set-Cookie:lidc="b=VB87:g=518:u=60:i=1477649823:t=1477731496:s=AQG-LTdly5mcIjAtiRHIOrKE1TiRWW-l"; Expires=Sat, 29 Oct 2016 08:58:16 GMT; domain=.thedomain.com; Path=/ Set-Cookie:_lipt=deleteMe; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/ Strict-Transport-Security:max-age=0 Transfer-Encoding:chunked Vary:Accept-Encoding, Avail-Dictionary X-Content-Type-Options:nosniff X-Frame-Options:sameorigin X-FS-UUID:882b3366afaa811400a04937a92b0000 X-Li-Fabric:prod-lva1 X-Li-Pop:prod-tln1-scalable X-LI-UUID:iCszZq+qgRQAoEk3qSsAAA== X-XSS-Protection:1; mode=block 

Fiddler Launch Code:

  Fiddler.FiddlerApplication.AfterSessionComplete += FiddlerApplication_OnAfterSessionComplete; Fiddler.FiddlerApplication.BeforeResponse += delegate(Fiddler.Session oS) { oS.utilDecodeResponse(); }; Fiddler.FiddlerApplication.Startup(0, FiddlerCoreStartupFlags.Default); } 

I initially assumed that it was chunked / gzipped, so I added utilDecodeResponse (); to onBeforeResponse, which had no effect!

Just to cover all the bases, I also tried to manually decode responseBodyBytes in UTF-8, Unicode, Bigendian, etc. just in time, the type of response content was wrong And turned off javascript and loaded the page to prove that these are not some funky templating thingy that also didn't matter.

Any ideas?

UPDATE:

According to the information provided by the developer and NineBerry below the solution, the following is given:

To prevent the response encoded by SDCH, you can add a handler as follows:

  Fiddler.FiddlerApplication.BeforeRequest += delegate (Fiddler.Session oS) { oS.oRequest["Accept-Encoding"] = "gzip, deflate, br"; }; 

It should be noted that this is not suitable for everything, since you manually adjust the headers and do not check if the SDCH is present and then delete it, for my purposes this works fine, but to use the general functions of the violinist proxy, you need more logic here.

+5
source share
1 answer

Content encoding is shown as SDCH - Dictionary Sharing; therefore manual decoding of responseBodyBytes in UTF-8, Unicode, Bigendian, etc. will not work in this case.

Here you can find more information about SDCH - SDCH Ref 1 and SDCH Ref 2

Excerpts from the above site:

Dictionary sharing is a method of encoding content that was proposed back in 2008 by Google and implemented in Chrome and is supported by a number of Google servers. The full offer can be obtained here - https://lists.w3.org/Archives/Public/ietf-http-wg/2008JulSep/att-0441/Shared_Dictionary_Compression_over_HTTP.pdf . Instead of copying the contents of the document in this blog post, I will try to summarize briefly and concisely:
The whole idea of ​​the protocol is to reduce redundancy between HTTP connections. The amount of “general HTTP response data” is obviously significant - for example, you often see a website using common headers and footers across multiple HTML pages. If the client needs to store this general data locally in a "dictionary", the server will only need to instruct the client on how to restore the page using this dictionary.

+4
source

Source: https://habr.com/ru/post/1258935/


All Articles