Deployment Architecture

Pending doom with LARGE inputs.conf file

jcmaynard
Explorer

Background:

Not sure how many of you do your inputs.conf files this way, but this is how we do it for the time being. One inputs.conf for each index deployed from the Deployment Server. Now, some of them are becoming QUITE massive at this point and we're running into issues of entries stepping on each other and cancelling out entries. We've just been adding new entries to the bottom of the file and pushing them out.

How can I manage this? What would be the best practice for keeping these files clean and avoiding stepping all over entries and avoiding gaps in data flow for our customer?

Thanks for any and all help/suggestions!

ckurtz
Path Finder

I learned early on to have an individual app per "input type" (such as one for weblogs and another for RADIUS, for example). The Deployment Server then can push only the needed apps to the proper servers (web app to the webservers, for example).

The inputs.conf for each app stays small because they only references the specific sourcetypes in the app...but this does mean I have dozens and dozens of inputs.conf, but all nicely sequestered in their own apps. This prevents a lot of the problem you describe.

The added bonus is that because I can modify only the individual app (and hence the individual serverclass) I can restart individual server classes without having to restart the entire deployment server. At only 100 clients, this isn't a huge help, but at 500 I can see it being an issue.

HTH.

jcmaynard
Explorer

In a perfect world...;-)

Thanks for this! But, unfortunately, I'm stuck with what I have. Great info!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...