My goal is to create a multi-tenant environment for monitoring several groups of Windows Servers.
In other words, I’d like to index every group with a separate dedicated index.
The Splunk Universal Forwarder is installed on every Windows Server and its output is directed to my Splunk Enterprise.
My original idea was:
1. To create a Server Class for each group of servers
2. To specify a separate index to each Sever Class
For the 1st group everything went ok:
Windows Events Logs -> New ->
Select Forwarders: Here I selected servers for the 1st group (group1) -> Next
Select Source: Here I selected relevant event log channels (Application) -> Next
Input Setting: Here I selected an index for the 1st group (idx_group1)-> Review -> Save
As a result, the new deployment application was created for the Server Class:
[WinEventLog://Application] disabled = 0 index = idx_group1
Indeed, the events from the server are indexed by idx_group1!
However, when I did the same for the 2nd group, I’ve got an error:
Cannot create another input for the event log "Application", one already exists.
Splunk says “The event log monitor runs once for every event log input defined in Splunk.”
So my question is – how can I collect the events from several groups of servers, when each group is indexed by a dedicated index?
That's one of the reasons I don't like the Forwarder Management when using advanced configurations.
Try editing the serverclass.conf file manually:
Unfortunately, I didn't find a way to associate the serverclass with the index.
The approach Route specific events to a different index is described in
Does it work for events sent by a Forwarder? Should I have only one Server Class for all server in this case?
Not clear to me to what props.conf and transforms.conf can I apply it in case of Windows Events Log.
Will it overwrite the index setting in inputs.conf of the application?
Is there any limitation on number of stanza's in transforms.conf? (I expect dozens of groups and dozens of server in each group)
It might be easier to just create one serverclass for all the windows servers and then group by app. That is, every app will go to a different group of servers based on the whitelist/blacklist regex you write in your serverclass.conf. See this
For instance (based on Example 3 from the link above):
[global] # whitelist.0=* at the global level ensures that the machineTypesFilter attribute # invoked later will apply. whitelist.0=* [serverClass:WindowsMachineTypes] machineTypesFilter=windows-* [serverClass:WindowsMachineTypes:app:Group1_EventLogsApp] whitelist.0=YOURWHITELIST blacklist.0=YOURBLACKLIST
You need to make sure every app has its own inputs.conf and that intputs.conf uses a different index.
This should work just fine providing you don't deploy several apps to the same server, that is, all the whitelists/blacklists need to be mutually exclusive.
An alternative that might increase performance but will probably simplify deployment is as follows:
I did this, but got the following errors:
stanza=serverClass:vcpewin:app:serverappgroup1 property=whitelist.0 reason='unsupported at this level'
stanza=serverClass:vcpewin:app:serverappgroup2 property=whitelist.0 reason='unsupported at this level'
my serverclasses.conf is
[serverClass:vcpe_win] restartSplunkd = true [serverClass:vcpe_win:app:_server_app_group1] whitelist.0 = 10.20.4.213 [serverClass:vcpe_win:app:_server_app_group2] whitelist.0 = 10.20.4.214