We have certain applications that we monitor for crashes on the network, so what we're doing is we have a script which will monitor the process, record the PID and then place it into a log every few minutes. Splunk polls the log file and saves the logs in the database.
What I'd like to accomplish is to set up an alert, whereby if the PID changes for an app on a given machine that Splunk will send an alert email notifying the SOC that the application has restarted.
The log format is:
_time AppName PID SessionName SessionNo MemUsage source
I'd like to tell when AppName's (executable file name) PID (numerical value) changes for source (machine name) within _time but I can't seem to figure out the best approach.