Counting results with versus without a term and graphing number as percentage for each day

[gbolcer]

I have 2,894 events when I do a search for everything in my index.

When a do a search for a subset of things with an added search term, I get 49.

So for that time period of everything, I want to be able to show a 1.6% out of 100% graph.

Further, I want for each day (24 hour period) to calculate the number of times a term shows up in events divided by the total events to show up as a daily 0-100% graph.

Ideas?

[gkanapathy]

The most efficient way for a whole index would be:

mysearchterm 
| stats count 
| eval [ eventcount | rename count as totalcount | format "" "" "" "" "" "" ] 
| eval pct=100*(count/totalcount)

But this is hacky and skeevy and not generalizable to get daily counts and percentages

More generally, the best way to get your would be to do:

* | timechart span=1d 
    count as totalcount
    count(eval(searchmatch("mysearchterm"))) as count 
  | eval pct=100*(count/totalcount)

Of course, you could modify your the part in the eval(...) and use a different function if that's more suitable: http://www.splunk.com/base/Documentation/4.1.12/SearchReference/CommonEvalFunctions

[gkanapathy]

Additionally, if you're doing this frequently, you should considering building and storing a summary index of the daily event counts rather than having to retrieve and count every event each time your run the query.

[David]

I wouldn't think that this would scale particularly well, since you're going to be running a global search, and if you get to tens or hundreds of thousands of events, it'll get very slow. But for small numbers you could use:

search | stats count as TotalCount | appendcols [search foo | stats count as SubsetCount] | eval Perc = round((SubsetCount/TotalCount)*100,2) | fields Perc

For the daily graph, you could put the above in a summary index and then just query that. If you wanted to do it on the fly, you could go for:

search | timechart count as TotalCount span=1d | join _time [search foo | timechart count as SubsetCount span=1d] | eval Perc = round((SubsetCount/TotalCount)*100,2) | fields - TotalCount, SubsetCount

Hope that helps.