Alerting

How do I create an alarm which triggers if the value of a field changes over time?

josedgaravito
New Member

Hello,

I am trying to create an alarm if the value of a field changes over time.

The value corresponds to the serial number of a device and I want to know how it is possible to generate an alarm if the device's serial is modified or altered.

Thanks for your help.

Tags (2)
0 Karma

woodcock
Esteemed Legend

It can be as simple as this run every hour for the last hour:

index=YouShouldAlwaysSpecifyAnInex sourcetype=AndSourcetypeToo
| stats values(SERIAL) dc(SERIAL) AS SERIALcount BY device
| search SERIALcount>1
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Assuming that you can otherwise identify the device, then you can.

We're going to assume it's not just one device, but an entire class of devices that you are trying to monitor. We're also going to assume that there is a fixed key of some sort, and for this case we'll assume that it's the IP address. (It doesn't really matter what it is.) We'll assume you are running your job daily.


STEP ONE - create the first file

your search that gets the current IPkey  and Serial (for the last month or so)
| fields IPKey Serial
| dedup IPKey
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate
| outputlookup  mySavedSerials.csv

That created a list of the current values.

STEP TWO - This checks for changes...

your search that gets the current IPkey  and Serial  (last 24 hours)
| fields IPKey Serial
| dedup IPKey Serial
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate

| rename COMMENT as "add the prior information"
| inputlookup  append=t mySavedSerials.csv

| eventstats dc(Serial) as SerialCount by IPKey 
| where SerialCount > 1

That will let the two (or more) divergent records thru, so that will be your alert.


STEP THREE - This updates the file to match the most recent values ...

your search that gets the current IPkey  and Serial  (last 24 hours)
| fields IPKey Serial
| dedup IPKey 
| eval LastDate=strftime(relative_time(now(),"@d"),"%Y-%m-%d")
| table  IPKey Serial LastDate

| rename COMMENT as "add the prior information"
| inputlookup  append=t mySavedSerials.csv

| rename COMMENT as "get the earliest date for each IPKey/Serial combo, then keep the latest record for each IPKey"
| stats min(LastDate) as LastDate by IPKey Serial
| sort 0 IpKey - LastDate
| dedup IPKey
| outputlookup  mySavedSerials.csv

The above is one possible way of building the system. You could also set up the lookup as a permanent record of what the serial was at any given time, and the structure of these three jobs would change slightly. It all depends on what you mean.

0 Karma

josedgaravito
New Member

How can i join the 3 steps yo make the complete query

0 Karma

josedgaravito
New Member

If I enter this, the flag becomes 1 and the query works, but only for a hostname, if in my query I do not enter the name the result is as if the serial will change for all the hosts, as I can do to only reflect the value of the change and not of all the host?

index="index_name" Name="name_pc"
| streamstats current=true window=1 global=false latest(SerialNumber) as last_machine_serial, latest(DiskSerial) as last_disk_serial, latest(ProcSerial) as last_proc_serial
| dedup last_machine_serial,last_disk_serial, last_proc_serial
| search [inputlookup SavedSerials.csv | table SerialNumber, DiskSerial, ProcSerial]
| eval flag=if(SerialNumber!=last_machine_serial or DiskSerial!=last_disk_serial or ProcSerial!=last_proc_serial,1,0)
| table _time, Name, last_machine_serial, last_disk_serial, last_proc_serial ,flag

0 Karma

Sukisen1981
Champion

Possible but your requirement is stated very generically.
It is possible to compare each event (and therefore the device serial number filed value) with its value in the previous event, trigger an alert if the two values do not match

0 Karma

josedgaravito
New Member

How you could compare the field taking into account the previous event, assuming that the field name is SERIAL

0 Karma

Sukisen1981
Champion
 | streamstats current=f window=1 last(SERIAL) as PREV_SERIAL

Last field value of SERIAL will be stored in PREV_SERIAL

0 Karma

josedgaravito
New Member

If I enter this, the flag becomes 1 and the query works, but only for a hostname, if in my query I do not enter the name the result is as if the serial will change for all the hosts, as I can do to only reflect the value of the change and not of all the host?

index="index_name" Name="name_pc"
| streamstats current=true window=1 global=false latest(SerialNumber) as last_machine_serial, latest(DiskSerial) as last_disk_serial, latest(ProcSerial) as last_proc_serial
| dedup last_machine_serial,last_disk_serial, last_proc_serial
| search [inputlookup SavedSerials.csv | table SerialNumber, DiskSerial, ProcSerial]
| eval flag=if(SerialNumber!=last_machine_serial or DiskSerial!=last_disk_serial or ProcSerial!=last_proc_serial,1,0)
| table _time, Name, last_machine_serial, last_disk_serial, last_proc_serial ,flag

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...