topic Re: Can you help me with search time field extraction using props.conf & transforms.conf? in Getting Data In

Can you help me with search time field extraction using props.conf & transforms.conf?

conan311 — Tue, 29 Sep 2020 21:33:06 GMT

Hi splunk gurus,

I am new to Splunk and having some difficulty with a search time field extraction.

This is a sample log I would like to parse. (FIX log)

20181009-14:55:41.450 : 8=FIX.4.29=6435=034=295849=BLAH4252=20181009-14:55:41.44556=BLAH4210=115

Inputs.conf

[default]
host = ..servername..
index = ..blah.. 

[monitor://D:\test\FIX\fix2.log]
sourcetype = test-fix-logs2

transforms.conf

[test1]
REGEX = 8=(?P<tag_8>[^,]*?)\cA
FORMAT = tag_8::$1

[test2]
REGEX = \cA9=(?P<tag_9>[^,]*?)\cA
FORMAT = tag_9::$1

props.conf

[test-fix-logs2]
REPORT-class1 = test1
REPORT-class2 = test2

I have spent a few hours, but I couldn't find what could be potentially wrong as I don't see those two fields ('tag_8' & 'tag_9') in the search result in Splunk Web.

Can anyone please advise?

Re: Can you help me with search time field extraction using props.conf & transforms.conf?

harsmarvania57 — Fri, 12 Oct 2018 08:16:24 GMT

Hi @conan311,

You can achieve this extraction on search head with 2 different methods.

Only with props.conf

You can do below configuration only in props.conf and both fields will be extracted based on your regex.

[test-fix-logs2]
EXTRACT-testing = 8=(?P<tag_8>[^,]*?)\cA9=(?P<tag_9>[^,]*?)\cA

Combination of props.conf and transforms.conf

props.conf

[test-fix-logs2]
REPORT-testing = test1,test2

transforms.conf

[test1]
CLEAN_KEYS = 0
REGEX = 8=(?P<tag_8>[^,]*?)\cA

[test2]
CLEAN_KEYS = 0
REGEX = \cA9=(?P<tag_9>[^,]*?)\cA

Re: Can you help me with search time field extraction using props.conf & transforms.conf?

conan311 — Mon, 15 Oct 2018 23:47:37 GMT

@harsmarvania57 , thanks for your answer. It doesn't look like my sourcetype defined in inputs.confg ("test-fix-logs2") gets acknowledged. Can you see what could be wrong?

Re: Can you help me with search time field extraction using props.conf & transforms.conf?

harsmarvania57 — Tue, 16 Oct 2018 07:57:12 GMT

Can you please clarify "acknowledged", are you able to see test-fix-logs2 sourcetype in splunk while running the splunk query ?

Additionally above props.conf and transforms.conf should be on Indexer or Heavy Forwarder whichever comes first from Universal Forwarder and then you need to restart splunk to take effect of those new configurations.