I would like to collect my windows perfmon data into a metrics index. Is this feature planned for the near future?
The reason: I've had very good experience with the new metrics index. Great performance and very powerfull mstats command (no extra data model plus acceleration jobs)
Thanks for your help in advance,
I do believe they will enable this feature for v.next but the problem is one of cost. The currently implementation of perfmon data uses CSV which has a significantly lower data cost than metrics which is billed by each metric.
I believe the intent is to address this.
Updated to include example
not on 7.x yet, so i can't try...but can your format the data on the way in to be in the right format for a metrics index?
For example, to get the required indexed metric fields:
[perfmon://CPU] counters = % Processor Time; % User Time; % Privileged Time disabled = 0 instances = * interval = 60 object = Processor useEnglishOnly=true index = tester sourcetype=perfmon:test
Props.conf (parsing layer):
[perfmon:test] TRANSFORMS-metric = cpu_metric TRANSFORMS-value = cpu_value
transforms.conf (parsing layer):
[cpu_metric] REGEX = collection=(.+)[\s\S]*counter=(.+)[\s\S]*instance=(.+) FORMAT = metric_name::$1.$3.$2 WRITE_META = true [cpu_value] REGEX = Value=(.+) FORMAT = metric_value::$1 WRITE_META = true
This is sort of what I'm hoping to try when we upgrade, but I'm still worried about all of the indexed fields, but hopefully the metrics index file is more efficient/smaller than tsidx?
I didn't mean at the forwarder, but at parse time. So the data gets sent to your indexer/hf, and then you do some index-time extractions to get the fields you need. I was hoping to try something similar once we get upgrade to 7.x
Admittedly, I've never really looked into the parse/index time config for perfmon data, but I feel like this could be doable?