Assemble many events into a particular visualization

[adrien_dereumau]

So I already have a set of data that I can access and on which I build a chart.

Under, you will find my actual result:

Here is the search request used:

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| stats list(duration) as "step duration", list(duration_s) as "step duration_s" by build_step, build_id
| sort build_id
| table build_id, build_step, "step duration", "step duration_s"

Here is a simple event, as an example:
{

build_step: Total build time
dataType: build_step_duration

duration: 0:05:52.954656
project: project 1

startTime: 11/06/2019, 16:30:03

variant: variant1
}

The problem is I can't really manage to make the visualization the way I would like. And I am not sure if what I want to achieve is even possible with the data in the format I have right now.

So, my objective here would be to have on the horizontal axis of the chart the build_id (as it is right now) but I want to make a stacked column chart with each of the build_steps's durations_s. In a more understandable depiction:

|-----------                  ------------
|
|step3_s
|________
|step2_s                   step4_s
|
|-----------
|step1_s                   ________
|                                step1_s
|______________________________________________________________________
|build_id_1               build_id_2      ...
|______________________________________________________________________

The problem I have here is that the build_step names are not the same for each build and will need to be found automatically so that their duration in seconds can be stacked.

[adrien_dereumau]

I finally created my own solution using:

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval project_variant = project.":".variant.":".startTime
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| chart avg(duration_s) as duration_avg_s over project_variant by build_step limit=0 cont=false

I was using the treillis format to show the project separately.
The problem is that the step names become easily unreadable as the number of different value rises.

Both your solution are better than mine as they display the step names way better than mine.

[woodcock · Accepted solution]

Try this (select column chart and set general to the middle stack mode😞

index=cc_esm3_monitoring dataType="build_step_duration" 
| eval duration= "0" + substr('duration',1,7)
| convert auto(duration) as duration_s
| eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
| chart limit=0 useother=f sum(duration_s) BY build_id build_step

Here is a run-anywhere example:

index=_*
| rename date_minute AS build_id
| stats count AS duration_s BY sourcetype build_id
| rename sourcetype AS build_step
| chart limit=0 useother=f sum(duration_s) AS "step duration_s" BY build_id build_step

[adrien_dereumau]

I tried both the solutions and yours is simpler and avoid some troubles with the lists.
The view is perfect once stacked bars.
Thanks a lot!

[to4kawa]

| makeresults
| eval raw="duration: 0:05:52.954656
duration: 0:06:52.954656
duration: 0:10:52.954656"
| makemv delim="
" raw
| mvexpand raw
| rex field=raw "(?<duration>\d.+)"
| eval duration=replace(duration,"\.\d+","")
| convert dur2sec(duration) as duration_s
| streamstats count
| eval chart_name="step_".count
| xyseries _time chart_name duration_s

Hi, this query is to verify the concept.

 index=cc_esm3_monitoring dataType="build_step_duration" 
 | eval duration=replace(duration,"\.\d+","")
 | convert auto(duration) as duration_s
 | eval build_id = tostring(project) + ":" + tostring(variant) + " " + tostring(startTime)
 | stats list(duration) as "step duration", list(duration_s) as "step duration_s" by build_step, build_id
 | sort build_id
 | table build_id, build_step, "step duration", "step duration_s"
 | xyseries build_id, build_step, "step duration_s"

How about it?

[adrien_dereumau]

Hi your answer is really good but I chose @woodcock answer because of its simplicity and the lists are finally messing with the graphs...
Thanks a lot for the answer and you allowed me to discover more about xyseries!