Splunk Search

I want to delete fields whose total value is less than the threshold on a timechart.

to4kawa
Ultra Champion
index=_internal
| eventstats count by sourcetype
| where count > 100
| timechart span=1m count by sourcetype

note:earliest=-60m

if the total is less than the threshold, I don't want to display the field.
eventstats calculate all events , which is inefficient.

Is there any other good way?

0 Karma
1 Solution

to4kawa
Ultra Champion
index=_internal
| timechart usenull=f useother=f limit=0 span=1m count by sourcetype
| addcoltotals labelfield=_time label=total
| transpose 0 header_field=_time
| where total >100
| transpose 0 column_name=_time header_field=column
| where _time!="total"

This is the fastest in normal SPL.
timechart option, usenull useother limit all are off, it is faster in this case. 

Thank you all.

Conclusion:

  1. Use tstats. It is the fastest. However, it may not be available.
  2. Use where clause. this is easy to understand and faster.
  3. To dashboard, @woodcock or @to4kawa 's solution is good. they are more faster and useful. Of course, you can use it ordinary.
  4. subsearch can reduce the number of searches.
  5. eventstats is slow. It is better to use another way.

@spayneort
your solution is good and fast.
I want to describe it more, so I made it like this. sorry.

View solution in original post

0 Karma

to4kawa
Ultra Champion
index=_internal
| timechart usenull=f useother=f limit=0 span=1m count by sourcetype
| addcoltotals labelfield=_time label=total
| transpose 0 header_field=_time
| where total >100
| transpose 0 column_name=_time header_field=column
| where _time!="total"

This is the fastest in normal SPL.
timechart option, usenull useother limit all are off, it is faster in this case. 

Thank you all.

Conclusion:

  1. Use tstats. It is the fastest. However, it may not be available.
  2. Use where clause. this is easy to understand and faster.
  3. To dashboard, @woodcock or @to4kawa 's solution is good. they are more faster and useful. Of course, you can use it ordinary.
  4. subsearch can reduce the number of searches.
  5. eventstats is slow. It is better to use another way.

@spayneort
your solution is good and fast.
I want to describe it more, so I made it like this. sorry.

0 Karma

woodcock
Esteemed Legend

Like this:

index=_internal
| timechart span=1m count by sourcetype
| rename _span AS span
| untable _time sourcetype count
| eventstats sum(count) AS sourcetype_count BY sourcetype
| where sourcetype_count > 100
| xyseries _time sourcetype count
| rename span AS _span

Or, even better:

|tstats count WHERE index=_internal BY sourcetype _time span=1m
| eventstats sum(count) AS sourcetype_count BY sourcetype
| where sourcetype_count > 100
| timechart span=1m count by sourcetype

to4kawa
Ultra Champion

thank you for reply, @woodcock
Certainly it is faster than timechart where.

faster:
eventstats <<< subsearch stats << timechart where < untable and xyseries

0 Karma

woodcock
Esteemed Legend

So I win then?

0 Karma

to4kawa
Ultra Champion

my query is faster a little .

0 Karma

darrenfuller
Contributor

Well...if we're counting by sourcetype, let me throw this into the ring:

| tstats count where index=_internal [| tstats count where index=_internal earliest=-60m by sourcetype 
                                      | search count > 100 
                                      | fields - count 
                                      | format] 
  by _time span=1m sourcetype
| timechart span=1m sum(count) as count by sourcetype

Mind you, this won't work if you are not working with an OOB Accelerated, or data model, or indexed extraction..ed field... but in this specific case it's marvelous.

./DF

to4kawa
Ultra Champion

thank you for reply, @darrenfuller
tstats is faster.
but it can't be used any time.

I will ask a little more.

0 Karma

spayneort
Contributor

Timechart with where clause

index=_internal earliest=-60m 
| timechart useother=false span=1m count by sourcetype WHERE count > 100

to4kawa
Ultra Champion

thank you for reply, @spayneort
Certainly it is faster than subsearch stats.

I will ask a little more.

faster:
eventstats < subsearch stats < timechart where

0 Karma

manjunathmeti
Champion

How about using subsearch? I found below query faster than one in the question.

index=_internal earliest=-60m 
    [ search index=_internal earliest=-60m 
    | stats count by sourcetype 
    | where count > 100 
    | fields sourcetype] 
| timechart span=1m count by sourcetype

to4kawa
Ultra Champion

thank you for reply, @manjunathmeti
Certainly it is faster than eventstats.

I will ask a little more.

faster:
eventstats < subsearch stats

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...