Pending doom with LARGE inputs.conf file

[jcmaynard]

Background:

Not sure how many of you do your inputs.conf files this way, but this is how we do it for the time being. One inputs.conf for each index deployed from the Deployment Server. Now, some of them are becoming QUITE massive at this point and we're running into issues of entries stepping on each other and cancelling out entries. We've just been adding new entries to the bottom of the file and pushing them out.

How can I manage this? What would be the best practice for keeping these files clean and avoiding stepping all over entries and avoiding gaps in data flow for our customer?

Thanks for any and all help/suggestions!

[ckurtz]

I learned early on to have an individual app per "input type" (such as one for weblogs and another for RADIUS, for example). The Deployment Server then can push only the needed apps to the proper servers (web app to the webservers, for example).

The inputs.conf for each app stays small because they only references the specific sourcetypes in the app...but this does mean I have dozens and dozens of inputs.conf, but all nicely sequestered in their own apps. This prevents a lot of the problem you describe.

The added bonus is that because I can modify only the individual app (and hence the individual serverclass) I can restart individual server classes without having to restart the entire deployment server. At only 100 clients, this isn't a huge help, but at 500 I can see it being an issue.

HTH.

[jcmaynard]

In a perfect world...;-)

Thanks for this! But, unfortunately, I'm stuck with what I have. Great info!