Splunk Search

How to sample first N minutes of span in timechart, with huge numbers of events?

rpecka
Explorer

I would like to run a timechart query that ends with `| timechart span=1h distinct_count(thing) by other_thing`

The problem is that there are a huge number of events being counted so the query takes a long time. Is there a way that I can run the same query but sample only the first 5 minutes of every hour so that I can speed up the query?

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rpecka,

you can use one of the answers of @bowesmana that are all correct or try adding:

time_minutes<6

to your main search.

Ciao.

Giuseppe

bowesmana
SplunkTrust
SplunkTrust

There are also a number of other possibilities to optimise a slow running query.

1. Check that the query is written efficiently. If you want to post your full query, some here may be able to suggest improvements

2. Run a regular saved search that collects the dc() every hour to a summary index and then use that data to report here

3. Run the search as a saved search and use that previously run search for your results.

4. If you really want the first 5 minutes, as opposed to sampling, you could then make a search that does this

search bla bla bla ((earliest=-4h@h latest=-4h@h+5m) OR 
                    (earliest=-3h@h latest=-3h@h+5m) OR 
                    (earliest=-2h@h latest=-2h@h+5m) OR 
                    (earliest=-h@h latest=-h@h+5m) OR
                    (earliest=@h latest=h+5m))

depending on whatever timerange you are including.

 

0 Karma

bowesmana
SplunkTrust
SplunkTrust

If this is in the search UI, you can set the Sampling ratio in the Event Sampling dropdown

bowesmana_0-1660623179839.png

Alternatively, if this is in a dashboard, set the sampleRatio XML element of the search

 

<search>
  <query>
  ... 
  </query>
  <sampleRatio>1</sampleRatio>
</search>

 

See the description here

https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Viz/PanelreferenceforSimplifiedXML#search

 

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...