Splunk Search

How would I build a table of all metrics and their dimensions in splunk

charliesfx
Explorer

My splunk server is receiving metrics from collectd.

I want to build a table showing the metrics, dimensions, and values emitted for each unique metric_name/dimension/host combination.

| metirc_name | dimension_1 | ... | dimension_n | avg_value over 10s | <- it would look something like that

Ideally what I'm asking for is something like this:

| mstats WHERE source=*, metric_name=*, index=* BY *  | table *
  • I want the value for each unique combination of metric_name and dimension and value for a given. But wildcards are not allowed in the BY clause.
  • I need to fill in empty dimensions because not all values will have the same dimensions.

I don't see a way to query with out explicitly defining each dimension key. Additionally I don't see a way to extract the raw datapoints from splunk. I could settle for averaging over a time span for the unique combination of host, metric_name, and all dimensions.

Any help is greatly appreciated!

Esky73
Builder

try :

| mcatalog values(_dims) WHERE "index"="*" GROUPBY metric_name index host
| rename values(_dims) AS dimensions 
| table metric_name dimensions index host

woodcock
Esteemed Legend

You need some combination of this:

| mcatalog values(_dims) WHERE index="*" AND sourcetype="*" BY index metric_name

And this:

| mstats latest(_value) WHERE index="*" AND sourcetype="*" AND metric_name="*" BY index metric_name

imgarytan
Path Finder

Ask the similar question ...

In Prometheus, there is such a concept called "all time series at current time", how can we achieve the same in Splunk?

0 Karma

rleviseur
Explorer

I'm having trouble figuring out how to access the dimensions and dimension values of metrics also. Plenty of documentation on how to list out the dimensions, but none for their values.

0 Karma

thaggie_splunk
Splunk Employee
Splunk Employee

I don't think there is a way to get all of the data out of a metrics index like this, what are you trying to achieve (why are you trying to do do this)?

If you have just 100 metrics, each with 5 dimensions, each with just 10 values that'd still be a table with 5,000 rows - that's more information than is appropriate to show a user in a table.

To list the dimensions and their values you use the mcatalog command:

| mcatalog values(_dims) WHERE metric_name=* AND index=*

| mcatalog values(your_dim) WHERE metric_name=* AND index=*

thaggie_splunk
Splunk Employee
Splunk Employee

"I could settle for averaging over a time span for the unique combination of host, metric_name, and all dimensions."

| mstats avg(_value) where index=* AND metric_name=* earliest=-1h latest=now by host, metric_name

charliesfx
Explorer

Thanks for replying, but this does not get me the unique values for host, metricname, and all dimensions. It only gives me unique values by host and metric name.

The problem I'm facing is that I want to group by each dimension, host, and metric name with out explicitly defining each dimension in the query.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...