Splunk Search

Is Splunk extracting unnecessary fields?

joshua_hart1
Path Finder

I've noticed that my searches are taking a very long time to complete. For instance, a one-hour search for Bro IDS events with sourcetype of bro_http takes 127 seconds to complete. When looking through the search.log, I see Splunk identifying valid field extraction stanzas for sourcetypes of all kinds--not just sourcetype=bro_http. When I use fast mode, the search takes all of a few seconds. Is there any way I can restrict the field extraction to only those stanzas that match the search terms?

Tags (2)
0 Karma

clanier
Explorer

I am experiencing similar issues, was there any answer or solution which was decided? I also made a post here: [Splunk Answers Question][1]
Any Insight would help!

https://answers.splunk.com/answers/586903/searchlog-indicates-search-is-attempting-to-extrac.html

0 Karma

mikaelbje
Motivator

Perhaps you have a props stanza for a wildcard source, sourcetype or host that gets applied to all sourcetypes? I.e.

[source::.*]
TRANSFORM-my_horrible_transform = my_horrible_transform

This stanza will also get executed together with any transforms you have for the bro_http sourcetype.

This may be an explanation if you have problems with slow searches for other sourcetypes too.

0 Karma

joshua_hart1
Path Finder

I'll check for that. Not sure I do, but I'll check. To be honest, I'm seeing stanza matches for Windows event logs when I search for Bro events or anything else. It's like Splunk loads every single props/transforms stanza for field extraction on every search regardless of the search criteria.

0 Karma

lmyrefelt
Builder

Since you say the sourcetype is bro_http i am guessing your data is full of key=value pairs? (or in json)
If so, splunk tries to extract all those per default.

you could in props.conf for sourcetype bro_http turn off automatic field extraction
[bro_http]
KV_MODE = none

However then you need to specify your own fields if you want to have fields searchable.

0 Karma

thomrs
Communicator

Use fast mode and be explicit about the fields you want.

Mode docs are here, http://docs.splunk.com/Documentation/Splunk/6.2.1/Search/Changethesearchmode.

More about the fields command, http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Fields.

0 Karma

joshua_hart1
Path Finder

I've read the mode documentation and I agree, fast mode is just as it's named: faster. However, when running inline searches for dashboard panels, searches often take a long time. Also, the root question about whether or not Splunk should attempt to extract fields based on field extraction stanzas that do not match the search terms remains to be answered.

For example, if my search is:

sourcetype=foo earliest=-60m@m

shouldn't Splunk only load the field extractions that match that source, sourcetype, or host? Currently, whenever I search for anything, Splunk loads the field extractions for everything under the sun. I believe this is leading to longer search times.

chanfoli
Builder

I too would like your original question addressed. I have extractions which are specified for a certain sourcetype, but see messages from splunk about them being slow, when my search is specific to a different sourcetype.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...