Splunk Search

How to correctly use mstats rate() without listing all dimensions in BY clause?

y0ft
Explorer

I have a monotonic counter metric named http_req and my metric data points are tagged with path and host dimensions, eg. path=/accounts or path=/transactions, host=server-1 or host=server-2.

Application running on each host starts with metric value 0 and increases the value after every request processed. If host is restarted, counting restarts from 0. Every application creates it's own metric data points.

I can create a visualization showing number of calls processed in time while grouping per path and discarding the host aspect using:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by path,host span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

I noticed that I need to specify all dimensions in BY clause of mstats for rate() to work correctly. Having known set of dimensions I can create proper query, but in the future more dimensions will be added.

Can this query be written in such way that listing all the dimensions is not necessary? eg. is there something like presented below possible?

| mstats ... by all-dimensions
0 Karma
1 Solution

y0ft
Explorer

I found a solution:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by _timeseries,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

Documentation around it could be improved. It is only mentioned in https://docs.splunk.com/Documentation/Splunk/8.0.1/Metrics/Histogramdatatype

View solution in original post

0 Karma

y0ft
Explorer

I found a solution:

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by _timeseries,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)

Documentation around it could be improved. It is only mentioned in https://docs.splunk.com/Documentation/Splunk/8.0.1/Metrics/Histogramdatatype

0 Karma

y0ft
Explorer

I found a workaround: add another dimension - tsid (time series identifier) - with random constant value for every counter.

| mstats rate(http_req) as http_req_rate WHERE index="metrices" by tsid,path span=1m
| stats sum(http_req_rate) by _time,path
| xyseries _time path sum(http_req_rate)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...