Splunk Search

Chart the actual value over time and not an average, etc.

msmapper
Path Finder

I know this should be a simple thing but I am trying to just chart out the trend of a value over time. I don't want an average or median or anything stats, i want the literal value that is being logged every 10secs or so. My log message is formatted in the following way

Feb 13 18:13:02 server01 PROCESS=enginedepth NAME=high VALUE=0

Feb 13 18:13:12 server01 PROCESS=enginedepth NAME=high VALUE=50

Feb 13 18:13:22 server01 PROCESS=enginedepth NAME=high VALUE=70

Feb 13 18:13:32 server01 PROCESS=enginedepth NAME=high VALUE=100

Feb 13 18:13:42 server01 PROCESS=enginedepth NAME=high VALUE=20

Feb 13 18:13:52 server01 PROCESS=enginedepth NAME=high VALUE=900

I am just looking to put number from the VALUE field in a chart for each NAME field.

The search query of "host=server01 PROCESS=enginedepth | timechart span=1m values(VALUE) by NAME limit=40 usenull=false useother=f" get close to what I am looking for but I get the additonal "total" line which throws off the charting. Plus I want to see all of the changes in the VALUE FIELD.

So my two questions are, is values() the correct stat to be used in this situation and is there a way to get rid of the auto-generated "total" field (green line in graph screenshot)?

alt text

1 Solution

somesoni2
Revered Legend

when you run the query in search, do you see any field named Total? if yes, then remove it using " | fields - total"

View solution in original post

yasinmoha
Path Finder

I tried using this query to check to see the trend of spool by time but it does not work.

source="syslog.txt" $HASP646 PROCESS=enginedepth | chart values(Spool) by _time TimeEx

Source Report looks like this.

8/6/15

12:00:00.970 AM
NR0000000 DCXA 15218 00:00:00.97 INTERNAL 00080A89 $HASP646 7.4688 PERCENT SPOOL UTILIZATION

Spool = 7.4688
TimeEx = 00:00:00

2   8/5/15

11:00:01.780 PM
NR0000000 DCXA 15217 23:00:01.78 INTERNAL 00000080 $HASP646 7.4050 PERCENT SPOOL UTILIZATION

Spool = 7.4050
TimeEx = 23:00:01

3   8/5/15

10:00:02.110 PM
NR0000000 DCXA 15217 22:00:02.11 INTERNAL 00080A89 $HASP646 9.0884 PERCENT SPOOL UTILIZATION

Spool = 9.0884
TimeEx = 22:00:02

4   8/5/15

9:00:01.910 PM

NR0000000 DCXA 15217 21:00:01.91 INTERNAL 00080A89 $HASP646 9.0335 PERCENT SPOOL UTILIZATION

Spool = 9.0335
TimeEx = 21:00:01

5   8/5/15

8:00:01.210 PM

NR0000000 DCXA 15217 20:00:01.21 INTERNAL 00080A89 $HASP646 8.9815 PERCENT SPOOL UTILIZATION

Spool = 8.9815
TimeEx = 20:00:01

6   8/5/15

7:00:01.620 PM

NR0000000 DCXA 15217 19:00:01.62 INTERNAL 00080A89 $HASP646 8.9438 PERCENT SPOOL UTILIZATION

Spool = 8.9438
TimeEx = 19:00:01

7   8/5/15

6:00:02.010 PM

NR0000000 DCXA 15217 18:00:02.01 INTERNAL 00000080 $HASP646 8.9057 PERCENT SPOOL UTILIZATION

Spool = 8.9057
TimeEx = 18:00:02
0 Karma

msmapper
Path Finder

This is exactly what i was looking for, I didn't think using |fields -total would remove the splunk generated field of total but it does. please submit it as an answer and i will accept it

0 Karma

somesoni2
Revered Legend

when you run the query in search, do you see any field named Total? if yes, then remove it using " | fields - total"

yannK
Splunk Employee
Splunk Employee

try to do a simple chart
<mysearch> PROCESS=enginedepth | chart values(VALUE) by _time NAME

or eventually add a bucketing |bucket _time span=10s before.

msmapper
Path Finder

yannK,

While your suggestion works just as well as the timechart query I had, it still doesn't provide a way to get rid of the "total" line that automatically gets generated. Is there any way to get rid of that? It totally skews the chart I am interested in the individual names not the totals.

0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...