topic Re: How do I highlight an event in the timeline? in Splunk Search

How do I highlight an event in the timeline?

safetytrick — Wed, 21 Mar 2018 19:44:49 GMT

I commonly need to find patterns within relation to a certain event. For instance I want to view all error logs after a code deploy, or I have a bug that causes a server crash and I want to look at the 30 minutes worth of logs before the crash.

In other tools I've used (Graphite, and New Relic) you can send special events for a code deploy and then those will be displayed in the graphs as a vertical line.

This feature in graphite is similar to what I want: graphite.readthedocs.io/en/latest/events.html

I tried using append with two queries:

This query finds the application startup: host="server-01.internal" WFLYSRV0025
This query finds shows me everything I'm looking for host="server-01.internal" ERROR

This was what I tried in append:
host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR]

However I can't differentiate between the regular search results and the event I want to highlight.

I've tried quite a few things and can't seem to figure out how to show the events I'm looking for?

Re: How do I highlight an event in the timeline?

mayurr98 — Thu, 22 Mar 2018 03:21:23 GMT

Can you try

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | highlight "ERROR"

Have a look at highlight SPL command.
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Highlight

let me know if this helps!

Re: How do I highlight an event in the timeline?

safetytrick — Thu, 22 Mar 2018 23:23:16 GMT

This highlights matches in the search results, what I'm hoping for is to show these special events in the timeline.

Re: How do I highlight an event in the timeline?

mayurr98 — Fri, 23 Mar 2018 01:11:28 GMT

Try this then

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | eval status=if(like(_raw,”%ERROR%”,”ERROR”,”NORMAL EVENTS”) | timechart span=1h count by status.

Let me know if this helps!

Re: How do I highlight an event in the timeline?

niketn — Fri, 23 Mar 2018 03:24:20 GMT

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

Re: How do I highlight an event in the timeline?

safetytrick — Fri, 23 Mar 2018 13:49:19 GMT

Thank you, yes this is exactly what I need. It would be nice to do this in the search, but this works too.