topic Re: Building a gantt chart in Splunk Search

Building a gantt chart

vbumgarn — Thu, 22 Apr 2010 00:23:44 GMT

I saw the output of dbinspect and how it's used in the "Index health" graph on the "Index status" dashboard. It looks pretty much like a gantt chart, which is how we would like to see client runs and how they overlap.

Given log messages like this, where there may be multiple entries with the same fullJobID that are all parts of the same "run":

2010-04-21 15:42:55,539 INFO  [Thread-3] () () () jobinfo Done: [client="foo", finished="2010-04-21 13:36:35", fullJobID="id.20100421_093039.foo.-.581473", started="2010-04-21 13:36:19"]

I ended up with this crazy search:

sourcetype="jobinfo" Done 
| convert timeformat="%Y-%m-%d %H:%M:%S.%N" mktime(started) as startedSeconds 
| convert timeformat="%Y-%m-%d %H:%M:%S.%N" mktime(finished) as finishedSeconds 
| stats max(finishedSeconds) as finishedSeconds min(startedSeconds) as startedSeconds by fullJobID client 
| sort - startedSeconds 
| eval runTime = finishedSeconds-startedSeconds 
| where runTime>300 
| streamstats count as runnum 
| eval combinedDate=startedSeconds.",".finishedSeconds 
| makemv delim="," combinedDate 
| mvexpand combinedDate 
| eval _time=combinedDate 
| timechart span=5m limit=0 first(runnum) by fullJobID

You end up with a point for start and end for each job at the height of the runnum provided by streamstats. Then setting Null values to "Connect", you get the desired result.

I'm wondering if there are some commands I don't know about that would eliminate a few of the steps in here. My combine/makemv/mvexpand combo seems a bit silly, for instance.

Cheers, Vincent

Re: Building a gantt chart

sideview — Fri, 30 Apr 2010 12:29:24 GMT

Nice. Your eval + makemv + mvexpand trick is probably the shortest way to split each single event with the startSeconds and endSeconds fields into two events.
while I can think of a couple considerably worse ways to get the same end result, i cant think of a better one. 😃

some tiny comments

1 your two convert commands can be combined into one. Whether the performance increase is significant i dont know, but it saves a little space.

| convert timeformat="%Y-%m-%d %H:%M:%S.%N" mktime(started) as startedSeconds mktime(finished) as finishedSeconds

2 limit is optional in timechart, and limit=0 is the default.

3 Super minor but one of your evals could be done with strcat instead. I only mention this because once in a blue moon eval will do something not entirely desired, like round large decimal numbers a little. In this case obviously it wouldnt matter, and possibly since they added the "." operator to eval this is ancient history anyway.

| strcat startedSeconds "," finishedSeconds combinedDate

Re: Building a gantt chart

kasu_praveen — Tue, 07 May 2013 06:43:27 GMT

Thank you. My requirement was bit more than this. I posted question in forum http://splunk-base.splunk.com/answers/86326/gantt-chart-in-splunk. Can you please look into this?

Re: Building a gantt chart

ChhayaV — Mon, 23 Sep 2013 05:00:17 GMT

hi
i don't have start and end seconds i have only process name and time it took to complete i want to make a gantt chart for the same .how can i go about it ?

Re: Building a gantt chart

sowings — Mon, 23 Sep 2013 16:46:21 GMT

If you have time it took to complete from a Splunk event you have _time which is the start time (epoch time, in seconds), so you have the end time, too. You may have to do some math from _time and the run time to get to end time, but once there, the above strategy will work for you.