I'm using Splunk to collect the state of Microsoft IIS web server app pools. I've noticed that when the Universal Forwarder collects Perfmon data that has instance names with spaces in, and when ingested into a Metrics index, that the instance name after the first space is lost. But this doesn't happen if I ingested into a normal index.
Here is my configuration in the inputs.conf file:
interval = 10
object = APP_POOL_WAS
counters = Current Application Pool State
instances = *
disabled = 0
index = metrics_index
sourcetype = perfmon:IISAppPoolState
It is on a machine which has IIS pools which have spaces in there names - ie "company website" "company portal" "HR web"
When this data is ingested into the metrics index and accessed via the following Splunk command:
| mstats latest(_value) as IISAppPoolState WHERE index=metrics_index metric_name="IISAppPoolState.Current Application Pool State" by instance, host
I end up with instance values that truncate at the first space. So "company website" becomes just "company" (and who knows what happens to "company portal").
However if I direct the data into a normal index the instance names are wrapped in quotes and the space in the instance name persevered.
Is there anyway to fix this behaviour? Collecting this data into a metrics index has worked fine until now but thanks to this server having IIS site names with spaces in them it's causing a real problem.