Splunk Forwarder Field Extractions from Source

[ifeldshteyn]

Hello,

I think I know the answer but just want to confirm it. I have a Universal Forwarder and want to extract a field from source and send it to the indexer. It's a regular log (not a CSV, PSV etc...) so I guess I cannot (?) use INDEXED_EXTRACTIONS.

inputs.conf
[monitor://C:\test\testname.log]
sourcetype = mytest

props.conf
[mytest]
EXTRACT-mytest= C:\test2\(?.+).log in source

If I add this settings to a non-forwarder splunk instance it works perfectly. I am able to extract mytest (testname in this example) variable from the source. If I copy the same settings to my universal forwarder it won't work, the data is forwarder but the field is not extracted. Do I have to convert it to a heavy forwarder? Or add these extractions to our indexers?

Was reading this --> https://answers.splunk.com/answers/155234/field-extractions-dont-work-for-forwarded-input-from-unive...

Based on the above, just making sure that Universal Forwarders are not able to extract fields from source unless they are certain file types like CSV, TSV etc... And if it is possible, can someone pass along an example with props ?

Thanks!

[vishaltaneja070]

@ifeldshteyn

Field extraction using Splunk can be done only at parsing phase not on input phase expect for some types like CSV, JSON etc.

So either HF in between UF and Indexer which can do the work. OR do the direct extraction on Indexer.

[woodcock]

Every inputs.conf file goes on the Forwarder (the place that contains/pulls-in the data). Whenever you are using EXTRACT- in props.conf, that needs to go on your Search Head (not Indexers as some have said).

[richgalloway]

Inputs.conf goes on your forwarders. props.conf goes on the indexers.

See https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

[ifeldshteyn]

Are you sure that's correct? I am currently using INDEXED_EXTRACTIONS stanza on CSV files defined in props.conf on a Universal Forwarder. It's able to extract csv column names and assign them the values in each row.

Example below..

inputs.conf
[monitor:///var/log/csvdata]
disabled = false
sourcetype = csvdata

props.conf
[csvdata]
SHOULD_LINEMERGE = false
REPORT-csvdata= csvdata_extracts
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER=1

transforms.conf
[csvdata_extracts]
DELIM = ","
FIELDS = RunDate,FilerName,UserName,events_sub_Count

[richgalloway]

Yes, I'm sure. Newer UFs can do some parsing, but the general rule is to put props.conf on the indexer. You may also need to put it on your search heads if it contains search-time properties (like REPORT-*).