Deployment Architecture

Pending doom with LARGE inputs.conf file

jcmaynard
Explorer

Background:

Not sure how many of you do your inputs.conf files this way, but this is how we do it for the time being. One inputs.conf for each index deployed from the Deployment Server. Now, some of them are becoming QUITE massive at this point and we're running into issues of entries stepping on each other and cancelling out entries. We've just been adding new entries to the bottom of the file and pushing them out.

How can I manage this? What would be the best practice for keeping these files clean and avoiding stepping all over entries and avoiding gaps in data flow for our customer?

Thanks for any and all help/suggestions!

ckurtz
Path Finder

I learned early on to have an individual app per "input type" (such as one for weblogs and another for RADIUS, for example). The Deployment Server then can push only the needed apps to the proper servers (web app to the webservers, for example).

The inputs.conf for each app stays small because they only references the specific sourcetypes in the app...but this does mean I have dozens and dozens of inputs.conf, but all nicely sequestered in their own apps. This prevents a lot of the problem you describe.

The added bonus is that because I can modify only the individual app (and hence the individual serverclass) I can restart individual server classes without having to restart the entire deployment server. At only 100 clients, this isn't a huge help, but at 500 I can see it being an issue.

HTH.

jcmaynard
Explorer

In a perfect world...;-)

Thanks for this! But, unfortunately, I'm stuck with what I have. Great info!

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...