Splunk Search

How to create alert if there is no data from an extracted field?

power12
Communicator

Hello Splunkers ,

I am trying to schedule an alert when there is no data from a particular field which is extracted field from last 30 minutes.

Below is the sample event

Feb 28 12:49:25 hostabc postfix/smtpd[61995]: connect from host1.abc.local[158.xx.xx]
Feb 28 12:49:25 hostxyz postfix/smtpd[61995]: connect from host2.abc.local[158.xx.xx.xx]
Feb 28 12:49:25 host123 postfix/smtpd[61995]: connect from host3.abc.local158.xx.xx.xxx]

I am using below regex to extract sourcehost which gives me host1.abc.local,host2.abc.local,host3.abc.local
| rex field=_raw ".*from (?<sourcehost>.*)" 

I want to create alert when I dont see any events for the last 30 minutes from the source hosts. The alert should say "No data received from <sourcehos> in the last 30 minutes"

Thanks in Advance

0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

You need to append a search which returns a list of sourcehosts you are interested in. Often people will store this list in a csv store and use inputlookup to retreive it

| append
[ | inputlookup sourcehostlist.csv

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or they could append the results of a search with a larger timeframe or a search of a summary index for example

| append
[ search index ... earliest=-10d latest=-1s
| dedup sourcehost
| fields sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or generate some events by hand each time

| append
[ | makeresults
  | fields - _time
  | eval sourcehost=split("host1,host1.abc.local,host2,host2.abc.local,host3,host3.abc.local",",")
  | mvexpand sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust

Splunk is good at finding data which is in the logs - it is not so good at finding data which isn't there.

In order to get around this, you need to provide Splunk with a list of everything you are looking for and then see if anything exists in your 30 minute period.

index=....
| rex "from (?<sourcehost>.*)"
| dedup sourcehost
| eval found=1
| append
  [ <list of sourcehosts you are looking for>
    | eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2
``` found = 1 if only in index, 2 if only in list, 3 if in both ```
0 Karma

power12
Communicator

@ITWhisperer  Thank you for your reply. I am trying to understand your search what should I do in my case in the search after the append


| append
[ host1.[abc.local],host2.[abc.local],host3.[abc.local]

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

 

I am getting error saying "missing a search command before

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You need to append a search which returns a list of sourcehosts you are interested in. Often people will store this list in a csv store and use inputlookup to retreive it

| append
[ | inputlookup sourcehostlist.csv

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or they could append the results of a search with a larger timeframe or a search of a summary index for example

| append
[ search index ... earliest=-10d latest=-1s
| dedup sourcehost
| fields sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

or generate some events by hand each time

| append
[ | makeresults
  | fields - _time
  | eval sourcehost=split("host1,host1.abc.local,host2,host2.abc.local,host3,host3.abc.local",",")
  | mvexpand sourcehost

| eval found=2 ]
| stats sum(found) as found by sourcehost
| where found = 2

power12
Communicator

@ITWhisperer Thank you that worked

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

 Prepare to elevate your security operations with the powerful upgrade to Splunk Enterprise Security 8.x! This ...

Get Early Access to AI Playbook Authoring: Apply for the Alpha Private Preview ...

Passionate about security automation? Apply now to our AI Playbook Authoring Alpha private preview ...

Reduce and Transform Your Firewall Data with Splunk Data Management

Managing high-volume firewall data has always been a challenge. Noisy events and verbose traffic logs often ...