Histogram/Chart Question

[rebourne]

Greetings, I am struggling to create a chart to show when our backups begin and end for each server. The purpose is to show how many over lapping backups are running at a single time so that we can stagger them as not to overload our NAS.

We are indexing messages for backups as such:

When it starts: host=hostname name=backup action=begin
When it ends: host=hostname name=backup action=end

I would like something similar to

Hostname:         Time
Host1:          [======]
Host2:        [=====]
Host3:                     [===]
etc.

That is my vision - a nice simple way to see when backups start/end for each host.

Any help or a point in the right direction would be much appreciated.

Thanks

[dmcguerty]

I wanted to do something similar. So very easy to to in HTML - and common. Wonder why this is such a roadblock for Splunk.
Thanks

[gkanapathy]

You probably want to combine transaction with concurrency

... | transaction host name startswith=("action=begin") endswith=("action=end") | concurrency duration=duration

This will list your backups, each with an additional field concurrency indicating the number of backups running at the start of that backup

[gkanapathy]

Oh I see, you want a concurrency chart like Gantt-type chart. Unfortunately, Splunk's charting modules don't display these easily with Splunk's data, and I haven't been able to come up with a good way to make it work. I suppose I'd just file an enhancement request. Note that Splunk's own dbinspect command displays a chart like what you want, but it does some ugly hacking to generate data to fit the display capabilities of the Splunk charting modules.

[rebourne]

I do see the duration. However when I graph the duration, it is graphing it as a value, not time. Is there a way to graph the duration over time? For example, action=begin would be at 1am and action=end would be at 2am. I would like to graph between 1am to 2am for host1, whereas host2 would be from 1:30am-2am.

[gkanapathy]

the transaction command adds a duration field to each transaction it assembles. Is that not what you need to see? So every transaction group will have _time and duration fields.

[rebourne]

Excellent! Thank you! I now have the duration of the events. Is there a way to have the duration show at the time that the event started? I am close with:

... | transaction host name startswith=("action=begin") endswith=("action=end") | concurrency duration=duration | timechart span=10m sum(duration) by host

This gets me close but the duration does not match up with the time. Ideas?

Thank you for your time!