We wonder if it makes sense to break down the
serverclass.conf into multiple smaller files.
As it grows into five or six thousand lines, we wonder if it makes sense to keep growing it up.
Very good question and absolutely is the answer. It all depends on how much automation you require for your organisation and how your load balancing & firewall is enabled.
Let's take a large organisation and assume the hosts/clients are part of CMDB. So the simple 1 level iteration we could do is
1. Collect hostname, operating_system from cmdb (into a CSV)
2. Using a script, split this into minimum of 2 serverclass . (eg Windows, Unix). Details of machineTypesFilter in here
So your script/automation can generate two serverclass (MYwindowsserverclass, MYnixserverclass) and put stanza automatically something like (example of
[serverClass:WindowsMachineTypes] machineTypesFilter=windows* whitelist.0=winhost1 whitelist.1=winhost2 .. whitelist.3012=winhostSomeOther3012 [serverClass:WindowsMachineTypes:app:AppsForDesktops] [serverClass:WindowsMachineTypes:app:AppsForWindowsOnly2]
This way, your Windows Serverclass App will contain settings for Windows and deployment-app to be pushed only for Windows etc.
You can split this further into more chunks depending on how much automation you need (like Win 2012, Win2008) on what you get from CMDB
The more modular you make, better the automation and management of whitelists
From the SE -
-- How many files of one type (inputs, props, transforms, server class) doesn't matter to Splunk, it's all read into memory as one object.
If breaking them into multiple files makes your job easier though, go for it. Doesn't matter to Splunk.