visualization of discrete events

[jgc94131]

I'd like to visualize some continuous time series data like 'response time' while displaying discrete events, like a deployment of code, on the same chart.

What is the best way to do this?

What is the best way to get discrete events into Splunk?

example:

http://etsycodeascraft.files.wordpress.com/2010/12/circles_launch3.png?w=500&h=322

[martin_mueller · Accepted solution]

I think this should be possible. Something like this for the search:

sourcetype=deployment OR sourcetype=response_time | timechart count(eval(sourcetype="deployment")) as Deployment avg(response_time) as "Response Time"

That's assuming your deployment ends up as a single event in the corresponding sourcetype, and that the other sourcetype has a field response_time to chart. Data sources could be different, for example using inputlookup on a CSV file of deployments.

The key part becomes the charting. Select a column chart, and through the formatting options tell Splunk to display the Response Time as an overlay with its own y-axis scale. Then you get vertical lines for each deployment and a line for your response time. Here's an example of how it would look using native Splunk 6.1 visualization:

(Open Image in another tab to view full size)

Query generating dummy data plus the timechart from above:

| gentimes start=-1 increment=30m | eval response_time = random()%100 + 50 | append [gentimes start=-1 increment=6h | eval sourcetype="deployment"] | rename starttime as _time | timechart count(eval(sourcetype="deployment")) as Deployment avg(response_time) as "Response Time"

[lguinn2]

AFAIK, you can't make a chart like this in "regular" Splunk - i.e., using simple XML or the built-in charting capabilities. I assume that you could do almost anything by converting a dashboard to HTML and using JavaScript (and the many graphics libraries) to get what you want.

Here is how I would get discrete or "marker" events into Splunk:

Option 1:

Create a text file and have Splunk monitor it like any other log file. I suggest that you format events in the file something like this:

2014-Jun-28 11:19:00 UTC label=SoftwareRelease version=4.3.2 product=WonderfulCode entered production

Option 2:

Create a CSV file and use it as a lookup. Here is a sample format:

timestamp,label,version,product,description
2014-Jun-28 11:19:00 UTC,SoftwareRelease,4.3.2,WonderfulCode,entered production