Getting Data In

Universal Forwarder and props.conf and transforms.conf

phoenixdigital
Builder

Just a quick question regarding the "Universal Forwarder"

I have setup my inputs.conf and outputs.conf in
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/local/
this works perfectly

However I also wanted to perform some processing on these inputs prior to sending to the indexer.

It made sense that I would need to add props.conf and transforms.conf to this directory.

This however did not appear to work. Adding the props.conf and transforms.conf files to the indexer worked however.

Is there a way to do this on the universal forwarder or does it need to be done on the indexer?

Tags (1)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

Universal Forwarder and Light Forwarder do not parse events before passing them on to the indexer. Because they do not, most props.conf and transforms.conf settings need to be done at the indexer. This is what makes these two Forwarders "lighter" than the standard "Heavy" forwarder and a Splunk indexer.

View solution in original post

irwinj_125
Explorer

Apologies for my ignorance.  I've having a similar issue.

Regarding this comment:

"This however did not appear to work. Adding the props.conf and transforms.conf files to the indexer worked however."

Exactly how do you add these files to the indexer?

0 Karma

DUThibault
Contributor

You can have your Universal Forwarder do the index-time work, meaning SEDCMD and TRANSFORMS, as well as sourcetyping. The trick is that the Universal Forwarder's props.conf and transforms.conf must be on the forwarder (if you edit them in /opt/splunk/etc/deployment-apps/_server_app_<forwarder_class>/local/, Splunk will send the files to the forwarders for you) and the props.conf [<sourcetype>] and [source::<source>] stanzas must have a force_local_processing = true clause. Note that if the Universal Forwarder does the indexing, the Splunk instances won't: all of the index-time work must be done on the Universal Forwarder.

dwaddle
SplunkTrust
SplunkTrust

Universal Forwarder and Light Forwarder do not parse events before passing them on to the indexer. Because they do not, most props.conf and transforms.conf settings need to be done at the indexer. This is what makes these two Forwarders "lighter" than the standard "Heavy" forwarder and a Splunk indexer.

walterk82
Path Finder

most props.conf and transforms.conf settings need to be done at the indexer

Is there a more comprehensive definition of "most" and "works"?

0 Karma

walterk82
Path Finder
0 Karma

phoenixdigital
Builder

Thanks for the information. Makes sense from the perspective of 'light' and 'heavy' system usage.

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...