topic Re: Splunk Forwarder Field Extractions from Source in Getting Data In

Splunk Forwarder Field Extractions from Source

ifeldshteyn — Wed, 26 Dec 2018 18:16:14 GMT

Hello,

I think I know the answer but just want to confirm it. I have a Universal Forwarder and want to extract a field from source and send it to the indexer. It's a regular log (not a CSV, PSV etc...) so I guess I cannot (?) use INDEXED_EXTRACTIONS.

inputs.conf
[monitor://C:\test\testname.log]
sourcetype = mytest

props.conf
[mytest]
EXTRACT-mytest= C:\test2\(?.+).log in source

If I add this settings to a non-forwarder splunk instance it works perfectly. I am able to extract mytest (testname in this example) variable from the source. If I copy the same settings to my universal forwarder it won't work, the data is forwarder but the field is not extracted. Do I have to convert it to a heavy forwarder? Or add these extractions to our indexers?

Was reading this --> https://answers.splunk.com/answers/155234/field-extractions-dont-work-for-forwarded-input-from-universal-forwarder.html?utm_source=typeahead&utm_medium=newquestion&utm_campaign=no_votes_sort_relev

Based on the above, just making sure that Universal Forwarders are not able to extract fields from source unless they are certain file types like CSV, TSV etc... And if it is possible, can someone pass along an example with props ?

Thanks!

Re: Splunk Forwarder Field Extractions from Source

richgalloway — Wed, 26 Dec 2018 18:55:04 GMT

Inputs.conf goes on your forwarders. props.conf goes on the indexers.

See https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

Re: Splunk Forwarder Field Extractions from Source

ifeldshteyn — Tue, 29 Sep 2020 22:32:20 GMT

Are you sure that's correct? I am currently using INDEXED_EXTRACTIONS stanza on CSV files defined in props.conf on a Universal Forwarder. It's able to extract csv column names and assign them the values in each row.

Example below..

inputs.conf
[monitor:///var/log/csvdata]
disabled = false
sourcetype = csvdata

props.conf
[csvdata]
SHOULD_LINEMERGE = false
REPORT-csvdata= csvdata_extracts
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER=1

transforms.conf
[csvdata_extracts]
DELIM = ","
FIELDS = RunDate,FilerName,UserName,events_sub_Count

Re: Splunk Forwarder Field Extractions from Source

richgalloway — Wed, 26 Dec 2018 21:28:36 GMT

Yes, I'm sure. Newer UFs can do some parsing, but the general rule is to put props.conf on the indexer. You may also need to put it on your search heads if it contains search-time properties (like REPORT-*).

Re: Splunk Forwarder Field Extractions from Source

woodcock — Mon, 11 Feb 2019 06:30:48 GMT

Every inputs.conf file goes on the Forwarder (the place that contains/pulls-in the data). Whenever you are using EXTRACT- in props.conf, that needs to go on your Search Head (not Indexers as some have said).

Re: Splunk Forwarder Field Extractions from Source

vishaltaneja070 — Mon, 11 Feb 2019 07:02:13 GMT

@ifeldshteyn

Field extraction using Splunk can be done only at parsing phase not on input phase expect for some types like CSV, JSON etc.

So either HF in between UF and Indexer which can do the work. OR do the direct extraction on Indexer.