How to search metrics index with mstats to aggregate by non metrics fields?

chips — Tue, 24 Nov 2020 17:16:09 GMT

I am using Splunk Add-on for Unix and Linux 8.2.0 and enabled metrics index to collect disk usage.

I can search the disk used percentage by below search but it is the average of all mount point.

| mstats avg(_value) where index=linux-os AND metric_name=df_metric.UsePct

If I only want to stats metrics for a specific mount point, it seems there is no way to do it with mstats command. Is there any other approach to do it by utlizing the metrics index fast performance?

By searching the raw data for the metrics index,

| msearch index=linux-os | search sourcetype=df_metric

the search result is like below which shows data was ingested in _json format and metrics are created in a separate metrics index. However, in metrics index, there is no way to differentiate by MountedOn field as it's not a "metrics".

{ [-] Filesystem: /dev/vda1 IP_address: 10.1.2.3 MountedOn: / OS_name: Linux Server OS_version: 3 Type: ext4 entity_type: TA_Nix environment: dev metric_name:df_metric.Avail_KB: 11035324 metric_name:df_metric.Size_KB: 20509408 metric_name:df_metric.UsePct: 44 metric_name:df_metric.Used_KB: 8429164 }

Any thoughts or solution?

Re: How to search metrics index with mstats to aggregate by non metrics fields?

chips — Tue, 24 Nov 2020 18:58:48 GMT

didn't realize I can use by clause on the non-metrics field.

|mstats avg(_value) where index=linux-os AND metric_name=df_metric.UsePct by MountedOn | where MountedOn="/opt"

topic How to search metrics index with mstats to aggregate by non metrics fields? in All Apps and Add-ons

How to search metrics index with mstats to aggregate by non metrics fields?

Re: How to search metrics index with mstats to aggregate by non metrics fields?