Splunk Search

Search Optimization

HeinzWaescher
Motivator

Hi,

I've got ~15.000 events where FieldA exists (in total there are 20.000.000 events). I want to filter out these events and I'm wondering about the performance of different approaches.

Why is

sourcetype=X AND FieldA=*

so slow compared to this

sourcetype=X AND FieldA

BR

Heinz

0 Karma
1 Solution

lukejadamec
Super Champion

In short, the first search takes much longer because it is searching for a lot more stuff eventhough the results are the same. Splunk reads the searches as follows:

sourtype AND X AND FieldA AND *

sourtype AND X AND FieldA

As for the thousands of events from an index of millions, I have found that creating a summary index of the _time and _raw data of the events I want to keep makes life a lot easier. Depending on your situation, you may find accelerated searches work better than summary searches.

View solution in original post

lukejadamec
Super Champion

In short, the first search takes much longer because it is searching for a lot more stuff eventhough the results are the same. Splunk reads the searches as follows:

sourtype AND X AND FieldA AND *

sourtype AND X AND FieldA

As for the thousands of events from an index of millions, I have found that creating a summary index of the _time and _raw data of the events I want to keep makes life a lot easier. Depending on your situation, you may find accelerated searches work better than summary searches.

HeinzWaescher
Motivator

Thanks a lot, now i've got it

0 Karma

Ayn
Legend

The first search doesn't even look for "FieldA", so the first "translated" search there should read:

sourcetype AND X AND *

Meaning Splunk won't look for the field name until after it's found all the values, to see if it can couple the searched value to a field called "FieldA". This is a very good thing to remember when constructing searches as obviously the more you can narrow them down the better.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...