Alerting

How do I create an alarm which triggers if the value of a field changes over time?

josedgaravito
New Member

Hello,

I am trying to create an alarm if the value of a field changes over time.

The value corresponds to the serial number of a device and I want to know how it is possible to generate an alarm if the device's serial is modified or altered.

Thanks for your help.

Tags (2)
0 Karma

woodcock
Esteemed Legend

It can be as simple as this run every hour for the last hour:

index=YouShouldAlwaysSpecifyAnInex sourcetype=AndSourcetypeToo
| stats values(SERIAL) dc(SERIAL) AS SERIALcount BY device
| search SERIALcount>1
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Assuming that you can otherwise identify the device, then you can.

We're going to assume it's not just one device, but an entire class of devices that you are trying to monitor. We're also going to assume that there is a fixed key of some sort, and for this case we'll assume that it's the IP address. (It doesn't really matter what it is.) We'll assume you are running your job daily.


STEP ONE - create the first file

your search that gets the current IPkey  and Serial (for the last month or so)
| fields IPKey Serial
| dedup IPKey
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate
| outputlookup  mySavedSerials.csv

That created a list of the current values.

STEP TWO - This checks for changes...

your search that gets the current IPkey  and Serial  (last 24 hours)
| fields IPKey Serial
| dedup IPKey Serial
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate

| rename COMMENT as "add the prior information"
| inputlookup  append=t mySavedSerials.csv

| eventstats dc(Serial) as SerialCount by IPKey 
| where SerialCount > 1

That will let the two (or more) divergent records thru, so that will be your alert.


STEP THREE - This updates the file to match the most recent values ...

your search that gets the current IPkey  and Serial  (last 24 hours)
| fields IPKey Serial
| dedup IPKey 
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate

| rename COMMENT as "add the prior information"
| inputlookup  append=t mySavedSerials.csv

| rename COMMENT as "get the earliest date for each IPKey/Serial combo, then keep the latest record for each IPKey"
| stats min(LastDate) as LastDate by IPKey Serial
| sort 0 IpKey - LastDate
| dedup IPKey
| outputlookup  mySavedSerials.csv

The above is one possible way of building the system. You could also set up the lookup as a permanent record of what the serial was at any given time, and the structure of these three jobs would change slightly. It all depends on what you mean.

0 Karma

josedgaravito
New Member

How can i join the 3 steps yo make the complete query

0 Karma

josedgaravito
New Member

If I enter this, the flag becomes 1 and the query works, but only for a hostname, if in my query I do not enter the name the result is as if the serial will change for all the hosts, as I can do to only reflect the value of the change and not of all the host?

index="index_name" Name="name_pc"
| streamstats current=true window=1 global=false latest(SerialNumber) as last_machine_serial, latest(DiskSerial) as last_disk_serial, latest(ProcSerial) as last_proc_serial
| dedup last_machine_serial,last_disk_serial, last_proc_serial
| search [inputlookup SavedSerials.csv | table SerialNumber, DiskSerial, ProcSerial]
| eval flag=if(SerialNumber!=last_machine_serial or DiskSerial!=last_disk_serial or ProcSerial!=last_proc_serial,1,0)
| table _time, Name, last_machine_serial, last_disk_serial, last_proc_serial ,flag

0 Karma

Sukisen1981
Champion

Possible but your requirement is stated very generically.
It is possible to compare each event (and therefore the device serial number filed value) with its value in the previous event, trigger an alert if the two values do not match

0 Karma

josedgaravito
New Member

How you could compare the field taking into account the previous event, assuming that the field name is SERIAL

0 Karma

Sukisen1981
Champion
 | streamstats current=f window=1 last(SERIAL) as PREV_SERIAL

Last field value of SERIAL will be stored in PREV_SERIAL

0 Karma

josedgaravito
New Member

If I enter this, the flag becomes 1 and the query works, but only for a hostname, if in my query I do not enter the name the result is as if the serial will change for all the hosts, as I can do to only reflect the value of the change and not of all the host?

index="index_name" Name="name_pc"
| streamstats current=true window=1 global=false latest(SerialNumber) as last_machine_serial, latest(DiskSerial) as last_disk_serial, latest(ProcSerial) as last_proc_serial
| dedup last_machine_serial,last_disk_serial, last_proc_serial
| search [inputlookup SavedSerials.csv | table SerialNumber, DiskSerial, ProcSerial]
| eval flag=if(SerialNumber!=last_machine_serial or DiskSerial!=last_disk_serial or ProcSerial!=last_proc_serial,1,0)
| table _time, Name, last_machine_serial, last_disk_serial, last_proc_serial ,flag

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...