Splunk Search

Why is it slower to write key=value when searching?

munang
Path Finder

A) index=main 192.168.172.10

B) index=main src_ip=192.168.172.10

 

I thought B) was faster.

Because the index is the same 

Based on lispy

A)The following is the case for.

[ AND 10 168 172 192 index::main ]


In this case, I need to find all the data with 192, 168, 172,10 in it, I thought this cost was higher.

 

B) in the case of

[ AND index::main [ OR src_ip::192.168.172.10 [ AND sourcetype::fgt_traffic [ OR srcip::192.168.172.10 [ AND 10 168 172 192 ] ] ] [ AND 10 168 172 192 ] [ AND sourcetype::xmlwineventlog:microsoft-windows-sysmon/operational [ OR sourceip::192.168.172.10 [ AND 10 168 172 192 ] ] ] ] ]

As a result, I think the "field extraction" function automatically recognizes the src_ip field and maps the corresponding sourcetype value, and generates lispy.

Q1) So shouldn't B) be faster because there is a sourcetype specified that needs to be explored that much?The actual search time was much faster in A).

Q2) And I'm curious about the process of lispy being created.

 

 

Thank you.

 
 
Tags (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Apparently src_ip is a field which is:

1) set as indexed field

2) is aliased from other fields in two sourcetypes.

This leads Splunk to look for the indexed fields. But since by default Splunk assumes the value of the indexed field is included in the raw message as well you're getting those ANDs of the separate tokens (that deals with the raw part) and the indexed field value (that's the :: token).

BTW, the fastest search in your case would be

index=main TERM(192.168.172.10)

BTW, check the specific stages' execution cost and reaults count.

0 Karma

tscroggins
Champion

Hi @munang,

Splunk is aware of fields through fields.conf, props.conf, and other settings.

Search A can be optimized with the TERM() function:

index=main TERM(192.168.172.10)

[ AND 192.168.172.10 index::main ]

All events with 192.168.1.172.10 as an indexed term will be returned.

If src_ip is an indexed field, search B can be optimized using the :: operator or by modifying fields.conf (see fields.conf.spec for more information):

index=main src_ip::192.168.172.10

[ AND index::main src_ip::192.168.172.10 ]

If src_ip is not an indexed field but 192.168.172.10 is an indexed term, you can optimize by combining the = operator and the TERM() function:

index=main src_ip=192.168.172.10 TERM(192.168.172.10)

The lispy will vary with your configured field extractions, field aliases, etc., but it will begin with the IP address as an indexed term:

[ AND 192.168.172.10 index::main [ ... ] ]

If the IP address is an indexed term but src_ip is not equal to the IP address, the event will be scanned but not returned.

Search terms are segmented similarly to indexed terms. See segmenters.conf.spec, https://help.splunk.com/en/splunk-enterprise/search/search-manual/10.2/search-primer/event-segmentat..., and https://help.splunk.com/en/splunk-enterprise/administer/manage-indexers-and-indexer-clusters/10.2/in... for more information.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...