Splunk Search

How do I highlight an event in the timeline?

safetytrick
Engager

I commonly need to find patterns within relation to a certain event. For instance I want to view all error logs after a code deploy, or I have a bug that causes a server crash and I want to look at the 30 minutes worth of logs before the crash.

In other tools I've used (Graphite, and New Relic) you can send special events for a code deploy and then those will be displayed in the graphs as a vertical line.

This feature in graphite is similar to what I want: graphite.readthedocs.io/en/latest/events.html

I tried using append with two queries:

This query finds the application startup: host="server-01.internal" WFLYSRV0025
This query finds shows me everything I'm looking for host="server-01.internal" ERROR

This was what I tried in append:
host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR]

However I can't differentiate between the regular search results and the event I want to highlight.

I've tried quite a few things and can't seem to figure out how to show the events I'm looking for?

Tags (1)
0 Karma
1 Solution

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

safetytrick
Engager

Thank you, yes this is exactly what I need. It would be nice to do this in the search, but this works too.

0 Karma

mayurr98
Super Champion

Can you try

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | highlight "ERROR"

Have a look at highlight SPL command.
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Highlight

let me know if this helps!

0 Karma

safetytrick
Engager

This highlights matches in the search results, what I'm hoping for is to show these special events in the timeline.

0 Karma

mayurr98
Super Champion

Try this then

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | eval status=if(like(_raw,”%ERROR%”,”ERROR”,”NORMAL EVENTS”) | timechart span=1h count by status. 

Let me know if this helps!

0 Karma
Get Updates on the Splunk Community!

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...