Splunk Search

How would I build a table of all metrics and their dimensions in splunk

charliesfx
Explorer

My splunk server is receiving metrics from collectd.

I want to build a table showing the metrics, dimensions, and values emitted for each unique metric_name/dimension/host combination.

| metirc_name | dimension_1 | ... | dimension_n | avg_value over 10s | <- it would look something like that

Ideally what I'm asking for is something like this:

| mstats WHERE source=*, metric_name=*, index=* BY *  | table *
  • I want the value for each unique combination of metric_name and dimension and value for a given. But wildcards are not allowed in the BY clause.
  • I need to fill in empty dimensions because not all values will have the same dimensions.

I don't see a way to query with out explicitly defining each dimension key. Additionally I don't see a way to extract the raw datapoints from splunk. I could settle for averaging over a time span for the unique combination of host, metric_name, and all dimensions.

Any help is greatly appreciated!

Esky73
Builder

try :

| mcatalog values(_dims) WHERE "index"="*" GROUPBY metric_name index host
| rename values(_dims) AS dimensions 
| table metric_name dimensions index host

woodcock
Esteemed Legend

You need some combination of this:

| mcatalog values(_dims) WHERE index="*" AND sourcetype="*" BY index metric_name

And this:

| mstats latest(_value) WHERE index="*" AND sourcetype="*" AND metric_name="*" BY index metric_name

imgarytan
Path Finder

Ask the similar question ...

In Prometheus, there is such a concept called "all time series at current time", how can we achieve the same in Splunk?

0 Karma

rleviseur
Explorer

I'm having trouble figuring out how to access the dimensions and dimension values of metrics also. Plenty of documentation on how to list out the dimensions, but none for their values.

0 Karma

thaggie_splunk
Splunk Employee
Splunk Employee

I don't think there is a way to get all of the data out of a metrics index like this, what are you trying to achieve (why are you trying to do do this)?

If you have just 100 metrics, each with 5 dimensions, each with just 10 values that'd still be a table with 5,000 rows - that's more information than is appropriate to show a user in a table.

To list the dimensions and their values you use the mcatalog command:

| mcatalog values(_dims) WHERE metric_name=* AND index=*

| mcatalog values(your_dim) WHERE metric_name=* AND index=*

thaggie_splunk
Splunk Employee
Splunk Employee

"I could settle for averaging over a time span for the unique combination of host, metric_name, and all dimensions."

| mstats avg(_value) where index=* AND metric_name=* earliest=-1h latest=now by host, metric_name

charliesfx
Explorer

Thanks for replying, but this does not get me the unique values for host, metricname, and all dimensions. It only gives me unique values by host and metric name.

The problem I'm facing is that I want to group by each dimension, host, and metric name with out explicitly defining each dimension in the query.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...