Splunk Search

How do I highlight an event in the timeline?

safetytrick
Engager

I commonly need to find patterns within relation to a certain event. For instance I want to view all error logs after a code deploy, or I have a bug that causes a server crash and I want to look at the 30 minutes worth of logs before the crash.

In other tools I've used (Graphite, and New Relic) you can send special events for a code deploy and then those will be displayed in the graphs as a vertical line.

This feature in graphite is similar to what I want: graphite.readthedocs.io/en/latest/events.html

I tried using append with two queries:

This query finds the application startup: host="server-01.internal" WFLYSRV0025
This query finds shows me everything I'm looking for host="server-01.internal" ERROR

This was what I tried in append:
host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR]

However I can't differentiate between the regular search results and the event I want to highlight.

I've tried quite a few things and can't seem to figure out how to show the events I'm looking for?

Tags (1)
0 Karma
1 Solution

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

safetytrick
Engager

Thank you, yes this is exactly what I need. It would be nice to do this in the search, but this works too.

0 Karma

mayurr98
Super Champion

Can you try

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | highlight "ERROR"

Have a look at highlight SPL command.
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Highlight

let me know if this helps!

0 Karma

safetytrick
Engager

This highlights matches in the search results, what I'm hoping for is to show these special events in the timeline.

0 Karma

mayurr98
Super Champion

Try this then

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | eval status=if(like(_raw,”%ERROR%”,”ERROR”,”NORMAL EVENTS”) | timechart span=1h count by status. 

Let me know if this helps!

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...