Splunk Search

Issue with field extraction using Props.conf & transforms.conf

SplunkDash
Motivator

Hello,

I have some issues with field extraction using props.conf and transforms.conf files. Sample data (3 sample events), my props.conf and transforms.conf files are provided below. I am getting events but field extraction is not responding (or getting events without extracted fields). Any recommendations would be highly appreciated. Thank you!

 

3 Sample Raw Events

tP1380158753BMFPG68006701522000000000000000000000000000000 Y1021210324010157DFTJ450757015#26I040

tP1380158753BMFPG68016702522000000000000000000000000000000 Y1023210324010156DFTJ450757015#25I040

tP1380158753BMFPG68026703522000000000000000000000000000000 Y1023210324010155DFTJ450757015#26I040

props.conf

[abcd:tests]

SHOULD_LINEMERGE = false

CHARSET=UTF-8

LINE_BREAKER = ([\r\n]+)

NO_BINARY_CHECK=true

TIME_PREFIX=[^"]{62}

TIME_FORMAT=%Y%m%d%H%M%S

REPORT-abcdCod6 = abcdCod6

 

transforms.conf

[abcdCod6]

REGEX =

(^)(?i)(?<a>.{1})(?i)(?<b>.{1})(?i)(?<e>.{10})(?i)(?<f>.{5})(?i)(?<g>(?:6))(?i)(?<h>.{8})(?i)(?<i>.{1})(?i)(?<j>.{1})(?i)(?<k>.{30})(?i)(?<l>.{1})(?i)(?<m>.{1})(?i)(?<n>.{2})(?i)(?<o>.{14})(?i)(?<p>.{4})(?i)(?<q>.{9})(?i)(?<r>.{1})(?i)(?<s>.{2})(?i)(?<t>.{1})(?i)(?<u>.{2})(?i)(?<v>.{1})

DEST_KEY = _raw

Labels (2)
0 Karma

SplunkDash
Motivator

Hello @richgalloway ,

Thank you so much again. Yes, it has and marked as bold.

tP1380158753BMFPG68006701522000000000000000000000000000000 Y1021202224010157DFTJ450757015#26I040

tP1380158753BMFPG68016702522000000000000000000000000000000 Y1022202224010156DFTJ450757015#25I040

tP1380158753BMFPG68026703522000000000000000000000000000000 Y1023202224010155DFTJ450757015#26I040

 

 

 

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The bolded text, Y10212022, is not even close to the format specified in TIME_FORMAT.

%Y%m%d%H%M%S needs 14 characters, but only 9 are bold.  Based on the format string, I expected to see "2023" somewhere in the event, but that's not the case.  What's more, the highlighted text is 3 fields (m,n,o) rather than the one or two fields we usually see for timestamps.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Make sure the sourcetype specified in inputs.conf matches the stanza name in props.conf. 

Confirm the search heads were restarted since the .conf files were modified.

---
If this reply helps you, Karma would be appreciated.
0 Karma

SplunkDash
Motivator

Hello @richgalloway 

Thank you for your respond.

All looked good, as you mentioned. But still field extractions are not working, getting events under that sourcetype without extracted fields. Are there any other recommendations?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Do we have the complete raw events?  I ask because the sample events do not appear to have timetamps in them even though the TIME_PREFIX and TIME_FORMAT settings are present.  Without a complete event, it's impossible to confirm the accuracy of the regular expression.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...