Splunk Search

How to re-use search query using HiddenPostProcess

dabarb1
Explorer

So I'm attempting to re-use the same search query results multiple times in the same advanced view for performance reasons, but its not giving me the data that I want.

So what I want is to re-use a search query result set for two different bar graphs that show the top occurrences of a certain field.

For example, with individual queries it would be:

(Graph 1) HiddenSearch search string: field1=blah | top field2
(Graph 2) HiddenSearch search string: field1=blah | top field3

But with HiddenSearch / HiddenPostProcess I ended up with the following (according to the tiny amount of documentation explaining this feature):

HiddenSearch search string: field1=blah | stats count by field2, field3
(Graph 1) HiddenPostProcess search string: top field2
(Graph 2) HiddenPostProcess search string: top field3

The results using this method end up with a very small number because it doesn't factor in all the 'counts' from the main query.

What I ultimately want is for the parent query to give me the results of 'field1=blah' so that a piped 'top' command actually works in getting all the relevant counts.

sideview
SplunkTrust
SplunkTrust

In the "UI Examples" app, make sure you read the view called "Using PostProcess on Dashboards".

In short, you need to change your postProcess searches from top field2, to stats sum(count) as count by field2. To get the same behavior as top you may want to use eval to make a percent field, and you may want to use the head command to return only the top 10 or 20 rows.

The difference is that the top command is just going to look at the individual rows it's given, look at the field values each row has for field2, and calculate the value that is present in the most incoming rows. Nowhere in there does the top command pay attention to the count field. count is just a field and triggers no special behavior. So top ignores the count completely. However stats sum(count) as count by field2 explicitly pays attention to these counts, so you'll get the correct counts at the end.

0 Karma

sideview
SplunkTrust
SplunkTrust

No, not with the core modules. I would start using Sideview Utils, which provides a number of modules that can help here. The two relevant modules here are ValueSetter and HTML. 1) You can use a ValueSetter module to glue the $search$ and $postProcess$ values together and url-encode them. Give ValueSetter a name of "redirectURL", set ValueSetter's "urlEncodeKeys" param to "search,postProcess", then set its "value" param to "$search$ $postProcess$". 2) Then use an HTML module downstream from the ValueSetter, to create a link, and then you can use the $redirectUrl$ token within that link.

0 Karma

dabarb1
Explorer

I see your point. Anyways the next issue I have is I added a 'view results' link using the 'ViewRedirectorLink' module, but now that I'm using 'HiddenPostProcess', it only shows the parent search query, not the parent + post-process query that I want. Is there a way to create a link to a specific search query?

0 Karma

sideview
SplunkTrust
SplunkTrust

It "appears" to work but it will probably quietly lose events when the search is run at a larger scale. Read the docs page that I pointed you to, and get to know stats. I think that the table command alone, on the end of a plain event search, may still suffer the silent truncation at 50,000. stats definitely will not. More importantly if you're just using table you're not compressing the results at all so it's bound to perform more poorly.

0 Karma

dabarb1
Explorer

Thanks for the answer. However as I was thinking about this more, I discovered the 'table' search command where I just put all the fields I care about for post-processing as arguments. I tried that as well and found it appears to work, and it allows me to still use the 'top' command. Using 'top' is a lot more clear to me than using 'stats' twice. Do you see any downside to this approach?

0 Karma
Get Updates on the Splunk Community!

Security Professional: Sharpen Your Defenses with These .conf25 Sessions

Sooooooooooo, guess what. .conf25 is almost here, and if you're on the Security Learning Path, this is your ...

First Steps with Splunk SOAR

Our first step was to gather a list of the playbooks we wanted and to sort them by priority.  Once this list ...

How To Build a Self-Service Observability Practice with Splunk Observability Cloud

If you’ve read our previous post on self-service observability, you already know what it is and why it ...