Splunk Search

How do I highlight an event in the timeline?

safetytrick
Engager

I commonly need to find patterns within relation to a certain event. For instance I want to view all error logs after a code deploy, or I have a bug that causes a server crash and I want to look at the 30 minutes worth of logs before the crash.

In other tools I've used (Graphite, and New Relic) you can send special events for a code deploy and then those will be displayed in the graphs as a vertical line.

This feature in graphite is similar to what I want: graphite.readthedocs.io/en/latest/events.html

I tried using append with two queries:

This query finds the application startup: host="server-01.internal" WFLYSRV0025
This query finds shows me everything I'm looking for host="server-01.internal" ERROR

This was what I tried in append:
host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR]

However I can't differentiate between the regular search results and the event I want to highlight.

I've tried quite a few things and can't seem to figure out how to show the events I'm looking for?

Tags (1)
0 Karma
1 Solution

niketnilay
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketnilay
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

safetytrick
Engager

Thank you, yes this is exactly what I need. It would be nice to do this in the search, but this works too.

0 Karma

mayurr98
SplunkTrust
SplunkTrust

Can you try

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | highlight "ERROR"

Have a look at highlight SPL command.
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Highlight

let me know if this helps!

0 Karma

safetytrick
Engager

This highlights matches in the search results, what I'm hoping for is to show these special events in the timeline.

0 Karma

mayurr98
SplunkTrust
SplunkTrust

Try this then

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | eval status=if(like(_raw,”%ERROR%”,”ERROR”,”NORMAL EVENTS”) | timechart span=1h count by status. 

Let me know if this helps!

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!