topic Re: Still lost - Where do props and transforms go?? in Splunk Search

Still lost - Where do props and transforms go??

aferone — Thu, 29 Mar 2018 17:32:17 GMT

We have data coming from a file on a Universal Forwarder that requires field extractions. The extractions are in a props.conf file with many EXTRACT commands.

Where does the props file go?

On the Universal Forwarder?

On the Heavy Forwarder? Our logs go from Universal Forwarder --> Heavy Forwarder ..> Indexers (clustered)

On the Search Head? We have Clustered Search Heads.

I've installed it EVERYWHERE, and I still can't get it working.

One other side note. The props.conf file I received was developed and tested in a 7.X environment. But our production is 6.X. Should that even matter?

Thanks!!

Re: Still lost - Where do props and transforms go??

FrankVl — Thu, 29 Mar 2018 17:36:16 GMT

EXTRACTS are search time, so that should go on the search heads.

If you have it there and it is not getting applied, are you sure the sourcetype that gets assigned to the data matches what the props.conf stanza is triggering on? Can you post relevant parts of your config to help us spot the error? Incl. some sample data to verify if your extract statements are defined correctly?

The version difference shouldn't be too much of an issue, unless you use some specific feature that is new in 7. But basic extract stuff should work.

Re: Still lost - Where do props and transforms go??

aferone — Thu, 29 Mar 2018 17:40:54 GMT

I can confirm that the sourcetype of the data does match the props.conf stanza.

Re: Still lost - Where do props and transforms go??

aferone — Thu, 29 Mar 2018 17:41:55 GMT

Re: Still lost - Where do props and transforms go??

aferone — Thu, 29 Mar 2018 17:42:11 GMT

I had to attach pictures as an answer below.

Re: Still lost - Where do props and transforms go??

FrankVl — Tue, 03 Apr 2018 07:20:37 GMT

Ok, so the sourcetype matches. Can you show the rest of the config (and some sample data) to enable us to help you troubleshoot?

Re: Still lost - Where do props and transforms go??

splunker12er — Tue, 03 Apr 2018 07:51:48 GMT

Ok the question where do I place my props.conf in my deployment ?

Refer to this link : wiki link

I see your case fall under the category 4 -
Universal/Light Forwarder → Heavy Forwarder → Indexer
Input → Parsing → Indexing, Search

Your props.conf files according to your parsing / indexing / searching config as per requirement - it needs to placed in Heavy forwarder & Indexer ( read the wiki link for each phases what are the config stanzas job

Re: Still lost - Where do props and transforms go??

FrankVl — Tue, 03 Apr 2018 08:12:46 GMT

He mentions he is troubleshooting the field extractions, which are defined as EXTRACT commands in props.conf. EXTRACT works at search time and as the wiki you refer to suggests, such config should go on the Search Heads. So suggesting him to place it on the HF and Indexer doesn't make much sense.

Re: Still lost - Where do props and transforms go??

jaracan — Tue, 03 Apr 2018 09:05:08 GMT

Put the props.conf in Search Head since you are using the EXTRACT parameter.

EXTRACT- = [| in ]
* Used to create extracted fields (search-time field extractions) that do
not reference transforms.conf stanzas.

Reference:
https://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf

Re: Still lost - Where do props and transforms go??

aferone — Tue, 03 Apr 2018 15:08:46 GMT

Thanks for the help! There was some critical info missing in the props.conf that I overlooked. This app was written by someone else, so I didn't look at it carefully enough apparently. But indeed, this config belongs on the search head.

Thanks again!

Re: Still lost - Where do props and transforms go??

FrankVl — Tue, 03 Apr 2018 15:23:49 GMT

You're welcome, glad to hear you were able to solve the issue. And thanks for marking the answer as accepted 🙂