topic Re: Issue with field extraction using Props.conf & transforms.conf in Splunk Search

Issue with field extraction using Props.conf & transforms.conf

SplunkDash — Sat, 01 Apr 2023 20:30:48 GMT

Hello,

I have some issues with field extraction using props.conf and transforms.conf files. Sample data (3 sample events), my props.conf and transforms.conf files are provided below. I am getting events but field extraction is not responding (or getting events without extracted fields). Any recommendations would be highly appreciated. Thank you!

3 Sample Raw Events

tP1380158753BMFPG68006701522000000000000000000000000000000 Y1021210324010157DFTJ450757015#26I040

tP1380158753BMFPG68016702522000000000000000000000000000000 Y1023210324010156DFTJ450757015#25I040

tP1380158753BMFPG68026703522000000000000000000000000000000 Y1023210324010155DFTJ450757015#26I040

props.conf

[abcd:tests]

SHOULD_LINEMERGE = false

CHARSET=UTF-8

LINE_BREAKER = ([\r\n]+)

NO_BINARY_CHECK=true

TIME_PREFIX=[^"]{62}

TIME_FORMAT=%Y%m%d%H%M%S

REPORT-abcdCod6 = abcdCod6

transforms.conf

[abcdCod6]

REGEX =

(^)(?i)(?<a>.{1})(?i)(?<b>.{1})(?i)(?<e>.{10})(?i)(?<f>.{5})(?i)(?<g>(?:6))(?i)(?<h>.{8})(?i)(?<i>.{1})(?i)(?<j>.{1})(?i)(?<k>.{30})(?i)(?<l>.{1})(?i)(?<m>.{1})(?i)(?<n>.{2})(?i)(?<o>.{14})(?i)(?<p>.{4})(?i)(?<q>.{9})(?i)(?<r>.{1})(?i)(?<s>.{2})(?i)(?<t>.{1})(?i)(?<u>.{2})(?i)(?<v>.{1})

DEST_KEY = _raw

Re: Issue with field extraction using Props.conf & transforms.conf

richgalloway — Sun, 02 Apr 2023 00:41:13 GMT

Make sure the sourcetype specified in inputs.conf matches the stanza name in props.conf.

Confirm the search heads were restarted since the .conf files were modified.

Re: Issue with field extraction using Props.conf & transforms.conf

SplunkDash — Sun, 02 Apr 2023 04:43:09 GMT

Hello @richgalloway

Thank you for your respond.

All looked good, as you mentioned. But still field extractions are not working, getting events under that sourcetype without extracted fields. Are there any other recommendations?

Re: Issue with field extraction using Props.conf & transforms.conf

richgalloway — Sun, 02 Apr 2023 20:22:21 GMT

Do we have the complete raw events? I ask because the sample events do not appear to have timetamps in them even though the TIME_PREFIX and TIME_FORMAT settings are present. Without a complete event, it's impossible to confirm the accuracy of the regular expression.

Re: Issue with field extraction using Props.conf & transforms.conf

SplunkDash — Sun, 02 Apr 2023 20:31:52 GMT

Hello @richgalloway ,

Thank you so much again. Yes, it has and marked as bold.

tP1380158753BMFPG68006701522000000000000000000000000000000 Y1021202224010157DFTJ450757015#26I040

tP1380158753BMFPG68016702522000000000000000000000000000000 Y1022202224010156DFTJ450757015#25I040

tP1380158753BMFPG68026703522000000000000000000000000000000 Y1023202224010155DFTJ450757015#26I040

Re: Issue with field extraction using Props.conf & transforms.conf

richgalloway — Mon, 03 Apr 2023 00:45:24 GMT

The bolded text, Y10212022, is not even close to the format specified in TIME_FORMAT.

%Y%m%d%H%M%S needs 14 characters, but only 9 are bold. Based on the format string, I expected to see "2023" somewhere in the event, but that's not the case. What's more, the highlighted text is 3 fields (m,n,o) rather than the one or two fields we usually see for timestamps.