I have a clean install of the newest GA Splunk + the universal forwarder on one windows server.
I did a basic setup, didn't configure anything than forwarding and receivings etc. basic stuff. I have a default configuration in Splunk_TA_windows which configures the destination index correctly as default for all things it monitors.
Almost all data is going to correct indexes, but I noticed some of the perfmon-data is in the default main index and nowhere else, for example these are in the main index:
[perfmon://CPU]
counters = % Processor Time
disabled = 0
instances = *
interval = 10
object = Processor
useEnglishOnly=true
index = perfmon
[perfmon://LogicalDisk]
counters = % Free Space; Free Megabytes
disabled = 0
instances = *
interval = 10
object = LogicalDisk
useEnglishOnly=true
index = perfmon
These collections:objects go to main index:
Available Memory:Memory
CPU Load: Processor
Network Interface:Network Interface
Free Disk Space:LogicalDisk
You can see this for example here:
... and here:
I noticed, that this seems to go like this:
object=Processor collection="CPU Load" --> main
object=Processor collection=CPU --> perfmon
object=LogicalDisk collection="Free Disk Space" --> main
object=LogicalDisk collection=LogicalDisk --> perfmon
... so the collection is different.
I confirm that I DO NOT have any other configurations that would change the index, no transforms, no nothing. The only place that configures these indexes is the inputs.conf in Splunk_TA_windows and it's default and correct.
Any idea why this happens? I did two times a totall clean install without any additional my own configurations and this happens.
I have all the indexes on the indexer, and also some of the data is going into these indexes so this is not about a missing index.
You will need to create the permon index on the indexer, and what you are seeing is normal behavior if an indexer receives data for an index it does not have - it will write it to the default index, which out of the box is 'main'