Splunk Search

How do I search for multiple errors found in /var/log/messages?

damonmanni
Path Finder

I want to search for the following 3 error combinations and send alert if any, some or all are found:

  1. Error #1 - process=kernel AND the strings "segfault" AND "error" appear
  2. Error #2 - process=abrt AND the string "core dump" appear
  3. Error #3 - process=xinetd AND the strings "EXIT" AND "omni" appear

My search attempt below seems to only find/match and report only Error #3 where I want to show any/all matches in the report.

My Current search is:

host=node-1 OR host=node-2 index=os
(source=/var/log/messages OR source=/var/log/secure sourcetype=syslog OR sourcetype=linux_secure (process=kernel AND segfault AND error) OR (process=abrt AND "core dump") OR (process=xinetd AND "EXIT" AND omni))
| dedup host
|stats count list(process), list(filesystem), list(event_time) by host
|rename host AS "NFS Server", list(process) AS "Failed Process", list(filesystem) AS "Failed Filesystem", count AS "Errors Found", list(event_time) as "Time"
|table "NFS Server", "Failed Process", "Failed Filesystem", "Errors Found", "Time"

All advice appreciated.
cheers,
D

Tags (1)
0 Karma

wildcats12
Explorer

It looks like you're limiting your results to 1 event per host with the dedup before the stats, which may be why you only see 1 error. If you remove that, do you see multiple error conditions by host?

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...