Splunk Search

Is Splunk extracting unnecessary fields?

joshua_hart1
Path Finder

I've noticed that my searches are taking a very long time to complete. For instance, a one-hour search for Bro IDS events with sourcetype of bro_http takes 127 seconds to complete. When looking through the search.log, I see Splunk identifying valid field extraction stanzas for sourcetypes of all kinds--not just sourcetype=bro_http. When I use fast mode, the search takes all of a few seconds. Is there any way I can restrict the field extraction to only those stanzas that match the search terms?

Tags (2)
0 Karma

clanier
Explorer

I am experiencing similar issues, was there any answer or solution which was decided? I also made a post here: [Splunk Answers Question][1]
Any Insight would help!

https://answers.splunk.com/answers/586903/searchlog-indicates-search-is-attempting-to-extrac.html

0 Karma

mikaelbje
Motivator

Perhaps you have a props stanza for a wildcard source, sourcetype or host that gets applied to all sourcetypes? I.e.

[source::.*]
TRANSFORM-my_horrible_transform = my_horrible_transform

This stanza will also get executed together with any transforms you have for the bro_http sourcetype.

This may be an explanation if you have problems with slow searches for other sourcetypes too.

0 Karma

joshua_hart1
Path Finder

I'll check for that. Not sure I do, but I'll check. To be honest, I'm seeing stanza matches for Windows event logs when I search for Bro events or anything else. It's like Splunk loads every single props/transforms stanza for field extraction on every search regardless of the search criteria.

0 Karma

lmyrefelt
Builder

Since you say the sourcetype is bro_http i am guessing your data is full of key=value pairs? (or in json)
If so, splunk tries to extract all those per default.

you could in props.conf for sourcetype bro_http turn off automatic field extraction
[bro_http]
KV_MODE = none

However then you need to specify your own fields if you want to have fields searchable.

0 Karma

thomrs
Communicator

Use fast mode and be explicit about the fields you want.

Mode docs are here, http://docs.splunk.com/Documentation/Splunk/6.2.1/Search/Changethesearchmode.

More about the fields command, http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Fields.

0 Karma

joshua_hart1
Path Finder

I've read the mode documentation and I agree, fast mode is just as it's named: faster. However, when running inline searches for dashboard panels, searches often take a long time. Also, the root question about whether or not Splunk should attempt to extract fields based on field extraction stanzas that do not match the search terms remains to be answered.

For example, if my search is:

sourcetype=foo earliest=-60m@m

shouldn't Splunk only load the field extractions that match that source, sourcetype, or host? Currently, whenever I search for anything, Splunk loads the field extractions for everything under the sun. I believe this is leading to longer search times.

chanfoli
Builder

I too would like your original question addressed. I have extractions which are specified for a certain sourcetype, but see messages from splunk about them being slow, when my search is specific to a different sourcetype.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...