Splunk Search

How to create alert if there is no data from an extracted field?

power12
Communicator

Hello Splunkers ,

I am trying to schedule an alert when there is no data from a particular field which is extracted field from last 30 minutes.

Below is the sample event

Feb 28 12:49:25 hostabc postfix/smtpd[61995]: connect from host1.abc.local[158.xx.xx]
Feb 28 12:49:25 hostxyz postfix/smtpd[61995]: connect from host2.abc.local[158.xx.xx.xx]
Feb 28 12:49:25 host123 postfix/smtpd[61995]: connect from host3.abc.local158.xx.xx.xxx]

I am using below regex to extract sourcehost which gives me host1.abc.local,host2.abc.local,host3.abc.local
| rex field=_raw ".*from (?<sourcehost>.*)" 

I want to create alert when I dont see any events for the last 30 minutes from the source hosts. The alert should say "No data received from <sourcehos> in the last 30 minutes"

Thanks in Advance

Labels (1)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

You need to append a search which returns a list of sourcehosts you are interested in. Often people will store this list in a csv store and use inputlookup to retreive it

| append
[ | inputlookup sourcehostlist.csv

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or they could append the results of a search with a larger timeframe or a search of a summary index for example

| append
[ search index ... earliest=-10d latest=-1s
| dedup sourcehost
| fields sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or generate some events by hand each time

| append
[ | makeresults
  | fields - _time
  | eval sourcehost=split("host1,host1.abc.local,host2,host2.abc.local,host3,host3.abc.local",",")
  | mvexpand sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust

Splunk is good at finding data which is in the logs - it is not so good at finding data which isn't there.

In order to get around this, you need to provide Splunk with a list of everything you are looking for and then see if anything exists in your 30 minute period.

index=....
| rex "from (?<sourcehost>.*)"
| dedup sourcehost
| eval found=1
| append
  [ <list of sourcehosts you are looking for>
    | eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2
``` found = 1 if only in index, 2 if only in list, 3 if in both ```
0 Karma

power12
Communicator

@ITWhisperer  Thank you for your reply. I am trying to understand your search what should I do in my case in the search after the append


| append
[ host1.[abc.local],host2.[abc.local],host3.[abc.local]

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

 

I am getting error saying "missing a search command before

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You need to append a search which returns a list of sourcehosts you are interested in. Often people will store this list in a csv store and use inputlookup to retreive it

| append
[ | inputlookup sourcehostlist.csv

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or they could append the results of a search with a larger timeframe or a search of a summary index for example

| append
[ search index ... earliest=-10d latest=-1s
| dedup sourcehost
| fields sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or generate some events by hand each time

| append
[ | makeresults
  | fields - _time
  | eval sourcehost=split("host1,host1.abc.local,host2,host2.abc.local,host3,host3.abc.local",",")
  | mvexpand sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

power12
Communicator

@ITWhisperer Thank you that worked

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...