Splunk Search

How do I highlight an event in the timeline?

safetytrick
Engager

I commonly need to find patterns within relation to a certain event. For instance I want to view all error logs after a code deploy, or I have a bug that causes a server crash and I want to look at the 30 minutes worth of logs before the crash.

In other tools I've used (Graphite, and New Relic) you can send special events for a code deploy and then those will be displayed in the graphs as a vertical line.

This feature in graphite is similar to what I want: graphite.readthedocs.io/en/latest/events.html

I tried using append with two queries:

This query finds the application startup: host="server-01.internal" WFLYSRV0025
This query finds shows me everything I'm looking for host="server-01.internal" ERROR

This was what I tried in append:
host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR]

However I can't differentiate between the regular search results and the event I want to highlight.

I've tried quite a few things and can't seem to figure out how to show the events I'm looking for?

Tags (1)
0 Karma
1 Solution

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

@safetytrick, I think your use case is for Event Annotation which is one of the new features of Splunk Enterprise 7 , Refer to the following answer and Splunk Documentation:

https://answers.splunk.com/answers/600749/chart-with-trendline-in-splunk.html
https://docs.splunk.com/Documentation/Splunk/latest/Viz/ChartEventAnnotations

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

safetytrick
Engager

Thank you, yes this is exactly what I need. It would be nice to do this in the search, but this works too.

0 Karma

mayurr98
Super Champion

Can you try

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | highlight "ERROR"

Have a look at highlight SPL command.
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Highlight

let me know if this helps!

0 Karma

safetytrick
Engager

This highlights matches in the search results, what I'm hoping for is to show these special events in the timeline.

0 Karma

mayurr98
Super Champion

Try this then

host="server-01.internal" WFLYSRV0025 | append [search host="server-01.internal" ERROR] | eval status=if(like(_raw,”%ERROR%”,”ERROR”,”NORMAL EVENTS”) | timechart span=1h count by status. 

Let me know if this helps!

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...