Splunk Search

Assemble many events into a particular visualization

adrien_dereumau
Path Finder

So I already have a set of data that I can access and on which I build a chart.

Under, you will find my actual result:
alt text

Here is the search request used:

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| stats list(duration) as "step duration", list(duration_s) as "step duration_s" by build_step, build_id
| sort build_id
| table build_id, build_step, "step duration", "step duration_s"

Here is a simple event, as an example:
{

build_step: Total build time
dataType: build_step_duration

duration: 0:05:52.954656
project: project 1

startTime: 11/06/2019, 16:30:03

variant: variant1
}

The problem is I can't really manage to make the visualization the way I would like. And I am not sure if what I want to achieve is even possible with the data in the format I have right now.

So, my objective here would be to have on the horizontal axis of the chart the build_id (as it is right now) but I want to make a stacked column chart with each of the build_steps's durations_s. In a more understandable depiction:

|-----------                  ------------
|
|step3_s
|________
|step2_s                   step4_s
|
|-----------
|step1_s                   ________
|                                step1_s
|______________________________________________________________________
|build_id_1               build_id_2      ...
|______________________________________________________________________

The problem I have here is that the build_step names are not the same for each build and will need to be found automatically so that their duration in seconds can be stacked.

Tags (3)
1 Solution

woodcock
Esteemed Legend

Try this (select column chart and set general to the middle stack mode😞

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| chart limit=0 useother=f sum(duration_s) BY build_id build_step

Here is a run-anywhere example:

index=_*
| rename date_minute AS build_id
| stats count AS duration_s BY sourcetype build_id
| rename sourcetype AS build_step
| chart limit=0 useother=f sum(duration_s) AS "step duration_s" BY build_id build_step

View solution in original post

adrien_dereumau
Path Finder

I finally created my own solution using:

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval project_variant = project.":".variant.":".startTime
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| chart avg(duration_s) as duration_avg_s over project_variant by build_step limit=0 cont=false

I was using the treillis format to show the project separately.
The problem is that the step names become easily unreadable as the number of different value rises.

Both your solution are better than mine as they display the step names way better than mine.

woodcock
Esteemed Legend

Try this (select column chart and set general to the middle stack mode😞

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| chart limit=0 useother=f sum(duration_s) BY build_id build_step

Here is a run-anywhere example:

index=_*
| rename date_minute AS build_id
| stats count AS duration_s BY sourcetype build_id
| rename sourcetype AS build_step
| chart limit=0 useother=f sum(duration_s) AS "step duration_s" BY build_id build_step

View solution in original post

adrien_dereumau
Path Finder

I tried both the solutions and yours is simpler and avoid some troubles with the lists.
The view is perfect once stacked bars.
Thanks a lot!

to4kawa
SplunkTrust
SplunkTrust
| makeresults
| eval raw="duration: 0:05:52.954656
duration: 0:06:52.954656
duration: 0:10:52.954656"
| makemv delim="
" raw
| mvexpand raw
| rex field=raw "(?<duration>\d.+)"
| eval duration=replace(duration,"\.\d+","")
| convert dur2sec(duration) as duration_s
| streamstats count
| eval chart_name="step_".count
| xyseries _time chart_name duration_s

Hi, this query is to verify the concept.

 index=cc_esm3_monitoring dataType="build_step_duration" 
 | eval duration=replace(duration,"\.\d+","")
 | convert auto(duration) as duration_s
 | eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
 | stats list(duration) as "step duration", list(duration_s) as "step duration_s" by build_step, build_id
 | sort build_id
 | table build_id, build_step, "step duration", "step duration_s"
 | xyseries build_id, build_step, "step duration_s"

How about it?

adrien_dereumau
Path Finder

Hi your answer is really good but I chose @woodcock answer because of its simplicity and the lists are finally messing with the graphs...
Thanks a lot for the answer and you allowed me to discover more about xyseries!

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!