Splunk Search

"Compress" a multiline search

atanasmitev
Path Finder

Hi,

I am trying to compress/optimize a search, spanning multiple lines, see below (obfuscated, but logically the same ). You can see it's hard to process, even in this form:

  index=myindex source="somesource" var1=* | stats count by var2 | search count > 50
| map maxsearches=100 search="search index=myindex source=\"somesource\" var2=$var2$ | stats dc(var1) as thisname by var2| where thisname > 10"
| appendcols
[ index=myindex source="somesource" var1=* | stats count by var2 | search count > 50
  | map maxsearches=100 search="search index=myindex source="somesource" var2=$var2$ | stats count(var1) as thatname by var2" ] 

Logically it looks like this:

PART1| stats dc(var1) as thisname by var2| where thisname > 10 | appendcols [PART1 | stats count(var1) as thatname by var2" ]

And should further be passed to "|table var2, thisname, thatname"

Is there a way, to optimize/compress the search to a form, similar to the one in bold above, perhaps saving PART1 and passing parameters to it ?

Tags (2)
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Simplifying that search directly I'd change it to this:

  index=myindex source="somesource" var1=* | stats count by var2 | search count > 50
| map maxsearches=100 search="search index=myindex source=\"somesource\" var2=$var2$ | stats dc(var1) as thisname count(var1) as thatname by var2| where thisname > 10"

However, I think you can change the approach entirely like this:

  index=myindex source=\"somesource\" [ search index=myindex source="somesource" var1=* | stats count by var2 | search count > 50 | fields var2 ]
| stats dc(var1) as thisname count(var1) as thatname by var2 | where thisname > 10

Depending on your data, you may even be able to do this:

index=myindex source=\"somesource\" var1=* | stats count dc(var1) as thisname count(var1) as thatname by var2 | where thisname > 10 AND count > 50 | fields - count

Note, the last approach does change things a tiny bit - before, the last searches didn't require var1=*. Whether this makes sense or not depends on your data and requirements.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Simplifying that search directly I'd change it to this:

  index=myindex source="somesource" var1=* | stats count by var2 | search count > 50
| map maxsearches=100 search="search index=myindex source=\"somesource\" var2=$var2$ | stats dc(var1) as thisname count(var1) as thatname by var2| where thisname > 10"

However, I think you can change the approach entirely like this:

  index=myindex source=\"somesource\" [ search index=myindex source="somesource" var1=* | stats count by var2 | search count > 50 | fields var2 ]
| stats dc(var1) as thisname count(var1) as thatname by var2 | where thisname > 10

Depending on your data, you may even be able to do this:

index=myindex source=\"somesource\" var1=* | stats count dc(var1) as thisname count(var1) as thatname by var2 | where thisname > 10 AND count > 50 | fields - count

Note, the last approach does change things a tiny bit - before, the last searches didn't require var1=*. Whether this makes sense or not depends on your data and requirements.

martin_mueller
SplunkTrust
SplunkTrust

You could say you wrote a separate program for printing out each of the letters in "Hello World!" - the original search had a separate search for each of the values in var2, and a separate search for each of the result columns 😛

atanasmitev
Path Finder

An unbiased colleague compared my version to a newbie 200-row "Hello world " search.

I tried the last one - WOW !!! Amazing ! It worked like a charm. Thanks a LOT!

Now I need to figure out how you did that, perhaps I should RTFM !!!

Thanks a lot !!!

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...