Getting Data In

How to use single inputs.conf across multiple forwarders with different set of monitored directories?

tusharsaran1
Path Finder

Hi All,

Is it possible to configure inputs.conf in such a way that universal forwarders running on different hosts can read the same file but scan a different set of directories? As an example, we want to create 1 inputs.conf with 3 stanzas . Now we want 3 different forwarders to read the same inputs.conf but monitor data from 1 directory each.
In other words, is it possible to link directory stanzas in inputs.conf with particular forwarders?

Our actual use case is given below:
We have a NFS mounted log directory having ~2000 subdirectories. We want to split the load across 4 universal forwarders with each forwarder scanning ~25% of the sub directories. We want to avoid managing 4 different inputs.conf files. Is that possible?

jonmargulies
Path Finder

There is no way to do what you want by just using one inputs.conf file. The closest I could think of would be to have one primary inputs.conf app that defines all the monitoring stanzas and disables them, and then a set of secondary inputs.conf apps, each assigned to just one forwarder, that just enables the appropriate stanzas for that forwarder. But I don't recommend doing that, as it's just another layer of complexity to manage (so now when you change something you have to change it in at least two places instead of one).

In the past, when I've had the need for very complex inputs.conf configurations, with hosts collecting data on behalf of thousands of other hosts, I've had a lot of success with building a spreadsheet that tracked all my inputs and then a Python script that processed the spreadsheet into the inputs.conf file(s) I needed. This had major benefits: I was just editing things in one place (the spreadsheet), which was a lot easier to read, sort, and search than an inputs.conf file; if I needed to change the way things were broken up, I could make the change in the Python script and have it automate the changes downstream; and much lower typo risk. I highly recommend this approach.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...