Getting Data In

Splunk Forwarder Field Extractions from Source

Path Finder

Hello,

I think I know the answer but just want to confirm it. I have a Universal Forwarder and want to extract a field from source and send it to the indexer. It's a regular log (not a CSV, PSV etc...) so I guess I cannot (?) use INDEXED_EXTRACTIONS.

inputs.conf
[monitor://C:\test\testname.log]
sourcetype = mytest

props.conf
[mytest]
EXTRACT-mytest= C:\test2\(?.+).log in source

If I add this settings to a non-forwarder splunk instance it works perfectly. I am able to extract mytest (testname in this example) variable from the source. If I copy the same settings to my universal forwarder it won't work, the data is forwarder but the field is not extracted. Do I have to convert it to a heavy forwarder? Or add these extractions to our indexers?

Was reading this --> https://answers.splunk.com/answers/155234/field-extractions-dont-work-for-forwarded-input-from-unive...

Based on the above, just making sure that Universal Forwarders are not able to extract fields from source unless they are certain file types like CSV, TSV etc... And if it is possible, can someone pass along an example with props ?

Thanks!

0 Karma

Motivator

@ifeldshteyn

Field extraction using Splunk can be done only at parsing phase not on input phase expect for some types like CSV, JSON etc.

So either HF in between UF and Indexer which can do the work. OR do the direct extraction on Indexer.

0 Karma

Esteemed Legend

Every inputs.conf file goes on the Forwarder (the place that contains/pulls-in the data). Whenever you are using EXTRACT- in props.conf, that needs to go on your Search Head (not Indexers as some have said).

0 Karma

SplunkTrust
SplunkTrust

Inputs.conf goes on your forwarders. props.conf goes on the indexers.

See https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

Are you sure that's correct? I am currently using INDEXED_EXTRACTIONS stanza on CSV files defined in props.conf on a Universal Forwarder. It's able to extract csv column names and assign them the values in each row.

Example below..

inputs.conf
[monitor:///var/log/csvdata]
disabled = false
sourcetype = csvdata

props.conf
[csvdata]
SHOULDLINEMERGE = false
REPORT-csvdata= csvdata
extracts
INDEXEDEXTRACTIONS = CSV
HEADER
FIELDLINENUMBER=1

transforms.conf
[csvdataextracts]
DELIM = ","
FIELDS = RunDate,FilerName,UserName,events
sub_Count

0 Karma

SplunkTrust
SplunkTrust

Yes, I'm sure. Newer UFs can do some parsing, but the general rule is to put props.conf on the indexer. You may also need to put it on your search heads if it contains search-time properties (like REPORT-*).

---
If this reply helps you, an upvote would be appreciated.
0 Karma