Splunk Search

How to correctly use mstats rate() without listing all dimensions in BY clause?

y0ft
Explorer

I have a monotonic counter metric named http_req and my metric data points are tagged with path and host dimensions, eg. path=/accounts or path=/transactions, host=server-1 or host=server-2.

Application running on each host starts with metric value 0 and increases the value after every request processed. If host is restarted, counting restarts from 0. Every application creates it's own metric data points.

I can create a visualization showing number of calls processed in time while grouping per path and discarding the host aspect using:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by path,host span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

I noticed that I need to specify all dimensions in BY clause of mstats for rate() to work correctly. Having known set of dimensions I can create proper query, but in the future more dimensions will be added.

Can this query be written in such way that listing all the dimensions is not necessary? eg. is there something like presented below possible?

| mstats ... by all-dimensions
0 Karma
1 Solution

y0ft
Explorer

I found a solution:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by _timeseries,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

Documentation around it could be improved. It is only mentioned in https://docs.splunk.com/Documentation/Splunk/8.0.1/Metrics/Histogramdatatype

View solution in original post

0 Karma

y0ft
Explorer

I found a solution:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by _timeseries,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

Documentation around it could be improved. It is only mentioned in https://docs.splunk.com/Documentation/Splunk/8.0.1/Metrics/Histogramdatatype

0 Karma

y0ft
Explorer

I found a workaround: add another dimension - tsid (time series identifier) - with random constant value for every counter.

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by tsid,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...