topic Re: How to Disable http response gzip encoding in Security

How to Disable http response gzip encoding

magnuspenilsson — Tue, 22 Mar 2011 03:06:27 GMT

I have a problem regarding gzip encoding. Any help is very appreciated.

The flash timelines and many charts does not work behind a proxy. The problem occurs when the http response is encoded with gzip, so the proxy can't rewrite the urls.

I have edited [web.conf] as follows: enable_gzip = False. Also, I put the Accept-Encoding: identity in the request header

Still no luck. No matter what I do, the response is gzip compressed (bug?)

... Example request and response fetching a timeline:

GET /en-US/app/search/flashtimeline?auto_pause=true&q=search HTTP/1.1

Host: localhost

Connection: keep-alive

Referer: http://localhost/SplunkProxy/en-US/app/search/dashboard

Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.98 Safari/534.13

Accept-Encoding: identity

Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.6,en;q=0.4

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Cookie: session_id_8000=e39121e162ef0cfc2d05e6c432b68307f918a0ed



HTTP/1.1 200 OK

Content-Type: text/html;charset=utf-8

Content-Encoding: gzip

Vary: Accept-Encoding

Server: Microsoft-IIS/7.5

Set-Cookie: session_id_8000=e39121e162ef0cfc2d05e6c432b68307f918a0ed; expires=Tue, 22 Mar 2011 14:35:00 GMT; Path=/

X-Powered-By: ASP.NET

Date: Mon, 21 Mar 2011 14:35:00 GMT

Content-Length: 15437



.............`.I.%&/m.{.J.J..t...`.$. ...... etc etc

Re: How to Disable http response gzip encoding

dwaddle — Tue, 22 Mar 2011 03:48:42 GMT

This appears to work for me under Splunk 4.1.7 on Linux:

$ cat /opt/splunk/etc/system/local/web.conf 
[settings]
mgmtHostPort = localhost:8099
enable_gzip = False

I checked with wireshark and do not see gzip compression. You should check your config with btool as follows:

/opt/splunk/bin/splunk cmd btool --debug web list | egrep -i "\[.*\]|gzip"
system     [default]
system     [settings]
system     enable_gzip = False

Re: How to Disable http response gzip encoding

magnuspenilsson — Tue, 22 Mar 2011 04:06:58 GMT

I have the same config.

Setting enable_gzip = True makes a lot of requests compressed, so it works as it is supposed to, excepts for timelines that is...

I am also using Wireshark. Have you tried to capture the exact same request? Default "Search" App Dashboard -> Click on a sourcetype to get to the search view. Examine the request in my example (GET /en-US/app/search/flashtimeline?auto_pause=true&q=search HTTP/1.1). In my case, it is compressed.

Re: How to Disable http response gzip encoding

Johnvey — Tue, 22 Mar 2011 04:20:08 GMT

Gzipping in HTTP occurs after the headers and should not affect any downstream servers.

The problem you are probably running into is the root endpoint setting. Looking at your request dump, you probably need to set the following in your web.conf file:

[settings]
root_endpoint = /SplunkProxy

This causes Splunkweb to forward all requests (whether proxied or direct) to this new sub-endpoint. Additional info is available in a previous post about proxying.

Background info on how HTTP gzipping works can be found in this tutorial.

Re: How to Disable http response gzip encoding

magnuspenilsson — Mon, 28 Sep 2020 09:26:26 GMT

Yes, gzipping occurs at the content only. IIS proxy (using ARR 2 & URL Rewrite 2) demands that the response is not gzipped for proxy reverse rewrite to work... Proxy is hosted on IIS at http://localhost/SplunkProxy/. Splunk is hosted at standard http://localhost:8000/. Then root_endpoint should be "/", right?. The GET parameter before proxy rewriting is "/en-US/app/search/flashtimeline?auto_pause=true&q=search HTTP/1.1". After it is "http:/localhost:8000/en-US/app/search/flashtimeline?auto_pause=true&q=search HTTP/1.1".

Re: How to Disable http response gzip encoding

magnuspenilsson — Tue, 22 Mar 2011 06:13:38 GMT

Solved. Johnvey, you are absolutely right about the root_endpoint. It should be set to "/SplunkProxy" if the proxy is located on "proxydomain/SplunkProxy/".

The reason for all my problems was that the IIS proxy using ARR and URL Rewrite must be configured at the proxy website root. I had an "Application" at localhost/SplunkTest where I also put the proxy rules which caused strange rewriting behaviors.

Thank you both of you.

Re: How to Disable http response gzip encoding

dwaddle — Wed, 23 Mar 2011 08:33:45 GMT

Make sure to mark Johnvey's answer as correct by clicking the green checkbox next to it. That will cause the answers system to treat this as 'answered' and help future users reading answers posts - not to mention hand out some rep points 🙂