Splunk Search

Hung Job Monitoring via Logs

Mick_OBrien
Path Finder

I have a search string that details the last log entry for all running jobs [shown in ascending order] bar a few jobs [as either completed OR of little interest]...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*")

What I now want is to reduce the returned set to logs that are over 30 mins old i.e. if the log file has NOT been updated in the past 30mins then there is a good chance that the job has hung

I have tried the following command...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*") | where earliest=-30m@m

...but I receive the following error...

"Error in 'where' command: The operator at 'm@m' is invalid."

Two questions

  1. What's wrong with my syntax and/or
  2. Is there a better way to to get the oldest last log entry from a set of job log files
Labels (1)
0 Karma

Mick_OBrien
Path Finder

Re-visited, re-thought and re-wrote.  This seems to work...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*") | where _time > relative_time(now(), "-1440m@m") AND _time < relative_time(now(), "-30m@m")

 I still do not understand what was wrong with original search string syntax

0 Karma
Get Updates on the Splunk Community!

Meet Duke Cyberwalker | A hero’s journey with Splunk

We like to say, the lightsaber is to Luke as Splunk is to Duke. Curious yet? Then read Eric Fusilero’s latest ...

The Future of Splunk Search is Here - See What’s New!

We’re excited to introduce two powerful new search features, now generally available for Splunk Cloud Platform ...

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...