topic Re: Caching All Static Content in the Splunk UI in Security

Caching All Static Content in the Splunk UI

ngift — Wed, 13 Oct 2010 05:41:09 GMT

In troubleshooting an unrelated problem with Firebug, I took a look at the amount of http calls to static content, and it seems fairly significant. What are the recommended approaches supported by Splunk to cache content?

When I have done Plone development in the past I have used Varnish [1] and that had a dramatic impact on a heavy UI. I am curious what caching system splunk customers use. We are currently using ngnix, so perhaps enabling their caching might be a good choice [2].

Re: Caching All Static Content in the Splunk UI

ziegfried — Wed, 13 Oct 2010 06:12:10 GMT

I've used Apache httpd in front of Splunk with mod_cache and mod_rewrite/mod_expires to cache static content both on the client side (Expires header) and on the reverse proxy (in-memory cache). This brought a noticeable performance gain in some cases. This is possible with nginx as well. If you need to optimize the UI performance, you could cache all URIs that start with either /<locale>/static or /<locale>/modules.

eg. /en-US/static/* and /en-US/modules/*

The only thing that you might be aware of is that some custom modules from splunkbase are generating static content on-the-fly. This generated content should not be cached and those modules might not work correctly with such a configuration.

As an example, the nginx configuration could look like this (not tested):

    location ~ ^/[a-z]+\-[A-Z]+/(static|modules)/.* {
        proxy_pass             http://127.0.0.1:8000;
        proxy_set_header       Host $host;
        proxy_cache            STATIC;
        proxy_cache_valid      200  1d;
        proxy_cache_use_stale  error timeout invalid_header updating
                               http_500 http_502 http_503 http_504;
        expires 24h;
    }

Re: Caching All Static Content in the Splunk UI

Simon — Sat, 01 Oct 2011 09:17:56 GMT

Another way would to cache only static objects matching a pattern, for example with varnish in vcl_recv:

if (req.request == "GET" && req.url ~ ".(gif|jpg|swf|css|js|png|jpg|jpeg|gif|png|tiff|tif|svg|swf|ico|css|js|vsd|doc|ppt|pps|xls|mp3|mp4|m4a|ogg|mov|avi|wmv|sxw|zip|gz|bz2|tgz|tar|rar|)$" ) {
return(lookup);
}

HTH.

Simon