topic Re: How to figure out if forwarders are utilizing props or transforms? in Getting Data In

How to figure out if forwarders are utilizing props or transforms?

tsheets13 — Wed, 28 Aug 2019 16:15:14 GMT

We have Universal Forwarder on our windows servers varying in version from 6.2.3 to 7.1.3. Our Splunk Enterprise version is 7.0.1 (upgrading soon).

I was always under the impression that formatting data on a UF was impossible but I have learned today that in some rare circumstances (structured data) that it can be done.

https://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime#Forwa

My question is, is there a way to tell with a search which, if any, forwarders are utilizing props or transforms?

Re: How to figure out if forwarders are utilizing props or transforms?

diogofgm — Wed, 28 Aug 2019 17:12:45 GMT

Hi tsheets13

Check this Wiki page. It contains a diagram of the indexing flow and where each conf file and/or conf attribute is used.
https://wiki.splunk.com/Community:HowIndexingWorks

Hope this helps clear some doubts.

Re: How to figure out if forwarders are utilizing props or transforms?

tsheets13 — Wed, 28 Aug 2019 17:16:50 GMT

Thanks but that doesn't really help. My objective is to determine if there are any formatting changes going on on the universal forwarders in our environment. We are planning upgrades and want to make sure we don't negatively affect anything. So I just need to determine if there are any of our UF's that have custom props or transforms running on them.

Re: How to figure out if forwarders are utilizing props or transforms?

diogofgm — Wed, 28 Aug 2019 17:42:10 GMT

you can use btool in CLI to determine what is being applied in your UF.

splunk btool props list --debug
AND
splunk btool transforms list --debug

anything that is not in system/default is somewhat "custom" and you can check the path of the "offender" .conf file

This can be used for all conf files (e.g server, web, etc.)

Re: How to figure out if forwarders are utilizing props or transforms?

tsheets13 — Wed, 28 Aug 2019 17:50:30 GMT

But these need to be run on the systems where the UF is installed, right? I was hoping there might be a way to tell from the searchhead.

Re: How to figure out if forwarders are utilizing props or transforms?

diogofgm — Wed, 28 Aug 2019 21:53:46 GMT

Yes. They need to be executed in the UF machines

Re: How to figure out if forwarders are utilizing props or transforms?

diogofgm — Wed, 28 Aug 2019 21:56:20 GMT

Following best pratices you would have most of the UF configs (if not all) managed by a deployment server. Leaving the other UF Config untouched. That way you could easily check what was being deployed just by looking into deployment apps.

Re: How to figure out if forwarders are utilizing props or transforms?

gcusello — Thu, 29 Aug 2019 06:45:40 GMT

Hi tsheets13,
as you said the only case where props and transforms are really used in UFs is ingesting structured data (e.g. csv).
But this is an advantage for you because you can manage these files in only one point (Indexers, Search Heads and Heavy Forwarders).
What is the reason
to use these files on UFs?
if you want to use them to filter logs, you can do (only wineventlog) in inputs.conf.
I don't see any additional reason to parse logs on UFs.

In addition, how do you manage UFs?
using Deployment Server you have a full control of your UFs configurations.

Bye.
Giuseppe

Re: How to figure out if forwarders are utilizing props or transforms?

woodcock — Sun, 01 Sep 2019 20:38:31 GMT

If the input is using INDEXED_EXTRACTIONS then the field creation is happening on the UF, otherwise it is not.