Splunk Search

Feature Request: troubleshooting/debugging for field extraction config files

Justin_Grant
Contributor

[UPDATE: from the answer below, it sounds like what I'm looking for is not supported in the product today. I'm tacking on the feature-request tag accordingly.]

When creating field extractions by editing props.conf/transforms.conf, it often takes a few iterations and experiments with regexes and config settings before I get it working.

What's the best way to troubleshoot and debug these problems? My ideal would be a trace or debug facility which would tell me, for a particular input (e.g. an imported log file) info like the following:

  1. which config files were used to try to extract fields? (can use this to track down problems like a config file having the wrong permissions or being in the wrong folder)

  2. for each config file, which stanzas were usd by that input? (to spot problems like typos in stanza names or auto-detection of sourcetype gone awry)

  3. for each stanza, verbose output about what happened when processing. For example if CHECK_FOR_HEADER = true was set, which header fields were actually captured? If there's a regex for field extraction, then how many times that regex matched? (if it's zero, I'll know there may be a problem with the regex)

  4. and so on... I'm not sophisticated enough to know all the things I should be asking for here, only that such a critical part of the product needs better troubleshooting/debugging support.

Essentially I want a record of what splunk tried to do, so I'll know at what stage the extraction failed.

Is something like this possible with Splunk 4.0 today? If so, how?

gkanapathy
Splunk Employee
Splunk Employee

And linebreaks and sourcetype classifications and timestamp extrations.

0 Karma

jrodman
Splunk Employee
Splunk Employee

What exists now is primarily trial and error.

You can use btool to fold together the various files to see what was really in each stanza, eg.

splunk cmd btool props list

You can list a specific stanza:

splunk cmd btool props list mysourcetype

However you can only find out how a event would have been processed in its entirety by discovering all the source patterns and host patterns that could have matched, as well as the sourcetype, and manually reviewing what they might have done.

I filed an ER for exactly this functionality. Please do raise the priority.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...

Customer success is front and center at .conf25

Hi Splunkers, If you are not able to be at .conf25 in person, you can still learn about all the latest news ...