Getting Data In

Splunk Forwarder Field Extractions from Source

ifeldshteyn
Communicator

Hello,

I think I know the answer but just want to confirm it. I have a Universal Forwarder and want to extract a field from source and send it to the indexer. It's a regular log (not a CSV, PSV etc...) so I guess I cannot (?) use INDEXED_EXTRACTIONS.

inputs.conf
[monitor://C:\test\testname.log]
sourcetype = mytest

props.conf
[mytest]
EXTRACT-mytest= C:\test2\(?.+).log in source

If I add this settings to a non-forwarder splunk instance it works perfectly. I am able to extract mytest (testname in this example) variable from the source. If I copy the same settings to my universal forwarder it won't work, the data is forwarder but the field is not extracted. Do I have to convert it to a heavy forwarder? Or add these extractions to our indexers?

Was reading this --> https://answers.splunk.com/answers/155234/field-extractions-dont-work-for-forwarded-input-from-unive...

Based on the above, just making sure that Universal Forwarders are not able to extract fields from source unless they are certain file types like CSV, TSV etc... And if it is possible, can someone pass along an example with props ?

Thanks!

0 Karma

vishaltaneja070
Motivator

@ifeldshteyn

Field extraction using Splunk can be done only at parsing phase not on input phase expect for some types like CSV, JSON etc.

So either HF in between UF and Indexer which can do the work. OR do the direct extraction on Indexer.

0 Karma

woodcock
Esteemed Legend

Every inputs.conf file goes on the Forwarder (the place that contains/pulls-in the data). Whenever you are using EXTRACT- in props.conf, that needs to go on your Search Head (not Indexers as some have said).

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Inputs.conf goes on your forwarders. props.conf goes on the indexers.

See https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

---
If this reply helps you, Karma would be appreciated.
0 Karma

ifeldshteyn
Communicator

Are you sure that's correct? I am currently using INDEXED_EXTRACTIONS stanza on CSV files defined in props.conf on a Universal Forwarder. It's able to extract csv column names and assign them the values in each row.

Example below..

inputs.conf
[monitor:///var/log/csvdata]
disabled = false
sourcetype = csvdata

props.conf
[csvdata]
SHOULD_LINEMERGE = false
REPORT-csvdata= csvdata_extracts
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER=1

transforms.conf
[csvdata_extracts]
DELIM = ","
FIELDS = RunDate,FilerName,UserName,events_sub_Count

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, I'm sure. Newer UFs can do some parsing, but the general rule is to put props.conf on the indexer. You may also need to put it on your search heads if it contains search-time properties (like REPORT-*).

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...