Archive
Highlighted

Deviation Detection

Path Finder

Hello,

We have certain applications that we monitor for crashes on the network, so what we're doing is we have a script which will monitor the process, record the PID and then place it into a log every few minutes. Splunk polls the log file and saves the logs in the database.

What I'd like to accomplish is to set up an alert, whereby if the PID changes for an app on a given machine that Splunk will send an alert email notifying the SOC that the application has restarted.

The log format is:

_time AppName PID SessionName SessionNo MemUsage source

I'd like to tell when AppName's (executable file name) PID (numerical value) changes for source (machine name) within _time but I can't seem to figure out the best approach.

Thanks!

Ken

Tags (2)
0 Karma
Highlighted

Re: Deviation Detection

SplunkTrust
SplunkTrust

If you just want to know if the PID has changed in the selected time range you could do a stats dc(PID) by AppName - if that is >1 you have a changed PID.

0 Karma
Highlighted

Re: Deviation Detection

Path Finder

Thanks Martin, not so sure that would work because the data is from about 300 sources so the PID will be different for each source.

0 Karma
Highlighted

Re: Deviation Detection

SplunkTrust
SplunkTrust

Append the source to the PID then to make it unique-er.

0 Karma