Monitoring Splunk

Is my alert query is correct

LogUx
Motivator

I want to create an alert for one particular error. So what would be the exact spl i need to write? 

Error is not in the intersting field.So i used this one.

I did from my end :

index=os  source="/var/log/messages" | eval  new_error= "server is not responding"

Is the above search correct? If not then please provide me the correct one.

 

 

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust
Any query can become an alert. Once the search completes, click the "Save as" dropdown and select "Alert".
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

No, your search is not correct.  It fetches all events from the /var/log/messages file and creates a field in each event called "new_error".  This probably is not the goal.  

To give a working we query we must know the purpose for the alert.  What is it looking for?

Perhaps this will get you started.

index=os source="/var/log/messages" "server is not responding"
---
If this reply helps you, Karma would be appreciated.

LogUx
Motivator

@richgalloway yes i tried this also, its working. Beacuse server is generating this error after every 2 or 3 days not sure. So if you suggest, can i set this simple one query into the final alert ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Any query can become an alert. Once the search completes, click the "Save as" dropdown and select "Alert".
---
If this reply helps you, Karma would be appreciated.

anilchaithu
Builder

@LogUx 

the SPL you mentioned won't work. Assuming "new_error" field is not available

 

index=os  source="/var/log/messages" | eval new_error= if(like(_raw,"%server is not responding%"), "Yes", "No") | where new_error="Yes"

 

 you can also use search in place of where command.

Hope this helps.

LogUx
Motivator

@anilchaithu  The mentioned SPL is not working. 

0 Karma

anilchaithu
Builder

@LogUx 

The logic should work. Do you have events in the data with this error? If yes, the text "server is not responding" in the eval command of the logic should match as is to the event.

0 Karma

LogUx
Motivator

@anilchaithu Yes i have the event which occured 4 days ago for that error. When i run a simple search as @richgalloway suggest it capture that event. But when i used to create a field for that error as you also suggest, it is not capturing any event. 

0 Karma

LogUx
Motivator

@anilchaithu Yes this approach is also seems correct. I will try this one and let you know.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...

A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure

AI + Observability: AI Agents, LLMs, Apps, & Infrastructure The rapid evolution of artificial intelligence ...