Out of our deployement of about 1,000 UF clients, a handful of systems are reporting data to the wrong indexes -- even though they are clearly configured to point to the correct one.
Here's the observations:
Lookup:
Name: daniels
Address: 10.14.108.60
Green_windows Server Class defined as:
10.14.96.*, 10.14.104.*, 10.14.105.*, 10.14.106.*, 10.14.107.*, 10.14.108.* <-- note the 108.*
Red_windows server class:
10.14.120.*, 10.14.121.*, 10.14.112.*, 10.14.12.*
BUT, he's showing up with Red's configurations:
daniels
Apps
Red_base_config, Red_windows
Server Classes
Red_windows
But wait, there's more: he's actually sending logs to two indexes: Red_windows AND Orange_windows... (but not Green's)
Sanity check -- the server class for Orange_windows is configured as:
10.14.40.*, 10.14.56.*, 10.14.72.*, 10.14.62.*, 10.14.78.*, 10.14.64.*, 10.14.13.*
We've confirmed the packages being deployed all point to the correct indexes -- and others in the same range are actually working properly!
Client is a Windows 10 system, if that matters...
Thoughts?
Thanks!
Verify you have the correct serverclass defined for this client in the WebGUI "Forwarder Management".
For more details, you can run the following on the deployment server
splunk list deploy-clients > my_dep_clients.txt
And then view that file. It should help breakdown which serverclasses each client belongs to and where the apps are coming from.
You should blacklist that host int the Red and Orange serverclass definitions but I suspect that the real problem is either:
That host is not a Deployment Client of this Deployment Server (should be discernable in the DS GUI).
The Splunk service could not be restarted on that host; manually go there and restart it. If the service does not restart, the configuration changes will not be put into effect.
I think this is the most likely case, Michael.
Note -- looks like my asterisks (*) didn't show up there -- please don't consider that a typo in my config...
I fixed that.