Splunk Search

Is Splunk extracting unnecessary fields?

joshua_hart1
Path Finder

I've noticed that my searches are taking a very long time to complete. For instance, a one-hour search for Bro IDS events with sourcetype of bro_http takes 127 seconds to complete. When looking through the search.log, I see Splunk identifying valid field extraction stanzas for sourcetypes of all kinds--not just sourcetype=bro_http. When I use fast mode, the search takes all of a few seconds. Is there any way I can restrict the field extraction to only those stanzas that match the search terms?

Tags (2)
0 Karma

clanier
Explorer

I am experiencing similar issues, was there any answer or solution which was decided? I also made a post here: [Splunk Answers Question][1]
Any Insight would help!

https://answers.splunk.com/answers/586903/searchlog-indicates-search-is-attempting-to-extrac.html

0 Karma

mikaelbje
Motivator

Perhaps you have a props stanza for a wildcard source, sourcetype or host that gets applied to all sourcetypes? I.e.

[source::.*]
TRANSFORM-my_horrible_transform = my_horrible_transform

This stanza will also get executed together with any transforms you have for the bro_http sourcetype.

This may be an explanation if you have problems with slow searches for other sourcetypes too.

0 Karma

joshua_hart1
Path Finder

I'll check for that. Not sure I do, but I'll check. To be honest, I'm seeing stanza matches for Windows event logs when I search for Bro events or anything else. It's like Splunk loads every single props/transforms stanza for field extraction on every search regardless of the search criteria.

0 Karma

lmyrefelt
Builder

Since you say the sourcetype is bro_http i am guessing your data is full of key=value pairs? (or in json)
If so, splunk tries to extract all those per default.

you could in props.conf for sourcetype bro_http turn off automatic field extraction
[bro_http]
KV_MODE = none

However then you need to specify your own fields if you want to have fields searchable.

0 Karma

thomrs
Communicator

Use fast mode and be explicit about the fields you want.

Mode docs are here, http://docs.splunk.com/Documentation/Splunk/6.2.1/Search/Changethesearchmode.

More about the fields command, http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Fields.

0 Karma

joshua_hart1
Path Finder

I've read the mode documentation and I agree, fast mode is just as it's named: faster. However, when running inline searches for dashboard panels, searches often take a long time. Also, the root question about whether or not Splunk should attempt to extract fields based on field extraction stanzas that do not match the search terms remains to be answered.

For example, if my search is:

sourcetype=foo earliest=-60m@m

shouldn't Splunk only load the field extractions that match that source, sourcetype, or host? Currently, whenever I search for anything, Splunk loads the field extractions for everything under the sun. I believe this is leading to longer search times.

chanfoli
Builder

I too would like your original question addressed. I have extractions which are specified for a certain sourcetype, but see messages from splunk about them being slow, when my search is specific to a different sourcetype.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...