Getting Data In

How do you search metrics? (contradictory documentation)

sillingworth
Path Finder

The documentation appears to contradict itself on this. The mstats documentation tends to perform its functions on the metric name (eg "avg(this_is_my_metric_name)") and states:

The WHERE clause cannot filter by metric_name. metric_name filtering is performed based on the metric_name fields specified by the <stats-metric-term> argument.

but the Metrics documentation does the opposite in its examples - its stats-metric-term is "_value" rather than a metric name, and its where clause filters on the metric name:

| mstats avg(_value) WHERE index=mymetricdata AND metric_name=aws.ec2.CPUUtilization span=30s

So which is the truth, and why do the documents contradict each other, or am I misunderstanding something?

0 Karma
1 Solution

thaggie_splunk
Splunk Employee
Splunk Employee

mstats can be used in two ways, where the metric name is used in the aggregation and where it is used in the filter.
To group by time, you add a span.

These are equivalent:

| mstats avg(_value) WHERE metric_name="my.metric.name" AND index="my-metrics-index" span=10s
| mstats avg(my.metric.name) WHERE index="my-metrics-index" span=10s

View solution in original post

thaggie_splunk
Splunk Employee
Splunk Employee

mstats can be used in two ways, where the metric name is used in the aggregation and where it is used in the filter.
To group by time, you add a span.

These are equivalent:

| mstats avg(_value) WHERE metric_name="my.metric.name" AND index="my-metrics-index" span=10s
| mstats avg(my.metric.name) WHERE index="my-metrics-index" span=10s

sillingworth
Path Finder

So I should never put "by _time" - simply including a span does that automatically?

0 Karma

thaggie_splunk
Splunk Employee
Splunk Employee

That is correct

0 Karma

thaggie_splunk
Splunk Employee
Splunk Employee

A good way to learn the mstats command is to install the "Metrics Workspace" app https://splunkbase.splunk.com/app/4192/ and see what it does.

inventsekar
SplunkTrust
SplunkTrust

Good learning about "metrics" feature. thanks for sharing.

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
0 Karma

sillingworth
Path Finder

Great - I'll do that, as I'm still struggling to do anything more than a simple search (eg a time chart by host). Thanks.

0 Karma

sillingworth
Path Finder

Also, does anybody have an example of grouping by _time that actually works? I always get "Error in 'mstats' command: Metrics Processor: Specify the span argument to group events by time." even after including a span.

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...