topic Histogram/Chart Question in Splunk Search

Histogram/Chart Question

rebourne — Sat, 12 Mar 2011 05:33:47 GMT

Greetings, I am struggling to create a chart to show when our backups begin and end for each server. The purpose is to show how many over lapping backups are running at a single time so that we can stagger them as not to overload our NAS.

We are indexing messages for backups as such:

When it starts: host=hostname name=backup action=begin
When it ends: host=hostname name=backup action=end

I would like something similar to

Hostname: Time
Host1: [======]
Host2: [=====]
Host3: [===]
etc.

That is my vision - a nice simple way to see when backups start/end for each host.

Any help or a point in the right direction would be much appreciated.

Thanks

Re: Histogram/Chart Question

gkanapathy — Sat, 12 Mar 2011 09:25:49 GMT

You probably want to combine transaction with concurrency

... | transaction host name startswith=("action=begin") endswith=("action=end") | concurrency duration=duration

This will list your backups, each with an additional field concurrency indicating the number of backups running at the start of that backup

Re: Histogram/Chart Question

rebourne — Tue, 15 Mar 2011 00:34:12 GMT

Excellent! Thank you! I now have the duration of the events. Is there a way to have the duration show at the time that the event started? I am close with:

... | transaction host name startswith=("action=begin") endswith=("action=end") | concurrency duration=duration | timechart span=10m sum(duration) by host

This gets me close but the duration does not match up with the time. Ideas?

Thank you for your time!

Re: Histogram/Chart Question

gkanapathy — Tue, 15 Mar 2011 01:09:05 GMT

the transaction command adds a duration field to each transaction it assembles. Is that not what you need to see? So every transaction group will have _time and duration fields.

Re: Histogram/Chart Question

rebourne — Tue, 15 Mar 2011 02:37:22 GMT

I do see the duration. However when I graph the duration, it is graphing it as a value, not time. Is there a way to graph the duration over time? For example, action=begin would be at 1am and action=end would be at 2am. I would like to graph between 1am to 2am for host1, whereas host2 would be from 1:30am-2am.

Re: Histogram/Chart Question

gkanapathy — Tue, 15 Mar 2011 03:44:32 GMT

Oh I see, you want a concurrency chart like Gantt-type chart. Unfortunately, Splunk's charting modules don't display these easily with Splunk's data, and I haven't been able to come up with a good way to make it work. I suppose I'd just file an enhancement request. Note that Splunk's own dbinspect command displays a chart like what you want, but it does some ugly hacking to generate data to fit the display capabilities of the Splunk charting modules.

Re: Histogram/Chart Question

dmcguerty — Wed, 07 May 2014 21:38:14 GMT

I wanted to do something similar. So very easy to to in HTML - and common. Wonder why this is such a roadblock for Splunk.
Thanks