Splunk Search

How to track slow-running field extractions?

stanwin
Contributor

Hi All

I have had a really bad Field extractor bogging down my system (discovered it from search.log on indexer) , tuning it made my search upto 18x faster for that app 😐 It slowed down no only searches but data model acceleration/pivots etc obviosly

I was interested in generating a auditing report on all slow running Field extractor's which would help boost the system by quite a bit ( though not all searches may be so bad to give such a performance boost)

But I dont see above log for quite a long timeline which means its not logged ( debug needed?) or criteria is different.

splunk version is 6.2.3 Build 264376

koshyk
Super Champion

I don't know any specific apps which might do, other than trying enabling debug mode and testing it specifically.

What I would do is:
- Isolate the problematic app only and remove all other apps (good place to try-out is your DEV system).
- Enable some sample data using eventgen or copy from prod and Index it with the problematic app. (approx 1 million events)
- Within the same app (or a different app) in "local" directory, create props.conf,transforms.conf,eventtypes.conf,tags.conf (or if you have clue about the problematic .conf file, just use that file only)
- copy all the extracts and put the value of all keys as "HELLO-test" (or some hardcoded variable)
- Now run the speed test to see if it gives you the 18x speed. Ideally it should give , otherwise the problem is somewhere else (eg problem in index time extractions)
- If the above results comes quickly, that means definitely it is a search time extraction regex.
- Now split the extractions: eg. copy half of the key-value extractions into the "local" conf files and run the speed test. If speed reduces, then it is one within your block you just copied.
- Repeat this process until you identify the extraction

Another option is, to use online "regex101.com" website and use the "debugger" enabled mode and check number of iterations and time taken for each regex.

0 Karma

rjthibod
Champion

This would be very nice to have.

0 Karma

stanwin
Contributor

bump!

Any analytics to run to get worst performing field extractions..

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I've converted this to a comment because it's not an answer.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I've converted this to a question so it doesn't get lost in a year-old topic.

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...