topic Re: How do you use a source stanza under props.conf on a universal forwarder? in Getting Data In

How do you use a source stanza under props.conf on a universal forwarder?

eangus — Tue, 29 Jan 2019 01:10:59 GMT

I'm currently looking at deploying some changes to ease management of input files in our environment. I've confirmed that the only way to bring in multiple whitelisted files, and think them with a source type, is to use a source stanza under props.conf. From what I've read, and tested, the sources props.conf would have to run on the forwarder instead of the indexers.

Has anyone tested the affects of the source stanza and resource utilization on a forwarder?

Here is an example of our configs.

INPUTS.CONF

[default]
index = my_index

[monitor:///export2/MyApp/*/logs/]
whitelist = MyApp[^/]*\.log|perflog\.txt[^/]*
followSymlink = false
disabled = 0

PROPS.CONF

[source::.../MyApp*]
sourcetype = my_index:agent

[source::.../Auto*]
sourcetype = my_index:auto

[source::.../MyAppManager*]
sourcetype = my_index:manager

[source::.../MyAppWeb*]
sourcetype = my_index:web

[source::.../perflog.txt*]
sourcetype = my_index:perflog

Re: How do you use a source stanza under props.conf on a universal forwarder?

chrisyounger — Tue, 29 Jan 2019 06:00:13 GMT

Hi @eangus

I am not aware of any published details for the impact of a change this specific.
From my experience, I would be confident making this change and I would expect it would have an unmeasurable impact on the UF.

If you are concerned, you should test in a non-prod environment while closely monitoring the server health.
All the best.

Re: How do you use a source stanza under props.conf on a universal forwarder?

markusspitzli — Tue, 19 Feb 2019 11:42:25 GMT

I don't know if you have a performance impact by using your configuration.
I'm just confused why it should ease the management of input files. In the end you will have to write whitelists and props.conf source:// entries, what makes it more difficult to understand the configuration IMHO.

I would suggest to write a inputs.conf like this:

[default]
index = my_index

[monitor:///export2/MyApp/*/logs/MyApp.log]
sourcetype = my_index:agent
followSymlink = false

[monitor:///export2/MyApp/*/logs/MyAppManager*.log]
sourcetype = my_index:manager
followSymlink = false 

[monitor:///export2/MyApp/*/logs/MyAppWeb*.log]
sourcetype = my_index:web
followSymlink = false 

[monitor:///export2/MyApp/*/logs/perflog.txt*]
sourcetype = my_index:perflog
followSymlink = false