Getting Data In

How do you use a source stanza under props.conf on a universal forwarder?

eangus
New Member

I'm currently looking at deploying some changes to ease management of input files in our environment. I've confirmed that the only way to bring in multiple whitelisted files, and think them with a source type, is to use a source stanza under props.conf. From what I've read, and tested, the sources props.conf would have to run on the forwarder instead of the indexers.

Has anyone tested the affects of the source stanza and resource utilization on a forwarder?

Here is an example of our configs.

INPUTS.CONF

[default]
index = my_index

[monitor:///export2/MyApp/*/logs/]
whitelist = MyApp[^/]*\.log|perflog\.txt[^/]*
followSymlink = false
disabled = 0

PROPS.CONF

[source::.../MyApp*]
sourcetype = my_index:agent

[source::.../Auto*]
sourcetype = my_index:auto

[source::.../MyAppManager*]
sourcetype = my_index:manager

[source::.../MyAppWeb*]
sourcetype = my_index:web

[source::.../perflog.txt*]
sourcetype = my_index:perflog
0 Karma

markusspitzli
Communicator

I don't know if you have a performance impact by using your configuration.
I'm just confused why it should ease the management of input files. In the end you will have to write whitelists and props.conf source:// entries, what makes it more difficult to understand the configuration IMHO.

I would suggest to write a inputs.conf like this:

[default]
index = my_index

[monitor:///export2/MyApp/*/logs/MyApp.log]
sourcetype = my_index:agent
followSymlink = false

[monitor:///export2/MyApp/*/logs/MyAppManager*.log]
sourcetype = my_index:manager
followSymlink = false 

[monitor:///export2/MyApp/*/logs/MyAppWeb*.log]
sourcetype = my_index:web
followSymlink = false 

[monitor:///export2/MyApp/*/logs/perflog.txt*]
sourcetype = my_index:perflog
followSymlink = false 
0 Karma

chrisyounger
SplunkTrust
SplunkTrust

Hi @eangus

I am not aware of any published details for the impact of a change this specific.
From my experience, I would be confident making this change and I would expect it would have an unmeasurable impact on the UF.

If you are concerned, you should test in a non-prod environment while closely monitoring the server health.
All the best.

0 Karma
Get Updates on the Splunk Community!

Data Preparation Made Easy: SPL2 for Edge Processor

By now, you may have heard the exciting news that Edge Processor, the easy-to-use Splunk data preparation tool ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Tips & Tricks When Using Ingest Actions

Tune in to learn about:Large scale architecture when using Ingest ActionsRegEx performance considerations ...