Splunk Search

Hung Job Monitoring via Logs

Mick_OBrien
Path Finder

I have a search string that details the last log entry for all running jobs [shown in ascending order] bar a few jobs [as either completed OR of little interest]...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*")

What I now want is to reduce the returned set to logs that are over 30 mins old i.e. if the log file has NOT been updated in the past 30mins then there is a good chance that the job has hung

I have tried the following command...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*") | where earliest=-30m@m

...but I receive the following error...

"Error in 'where' command: The operator at 'm@m' is invalid."

Two questions

  1. What's wrong with my syntax and/or
  2. Is there a better way to to get the oldest last log entry from a set of job log files
Labels (1)
0 Karma

Mick_OBrien
Path Finder

Re-visited, re-thought and re-wrote.  This seems to work...

index=ee_rpa_uipath_platform AND OrganizationUnitID=19 | dedup RobotName | table User, RobotName, _time, Message | sort _time | search (NOT Message IN("*complete*", "*ended*") AND NOT RobotName="ALWORKER*") | where _time > relative_time(now(), "-1440m@m") AND _time < relative_time(now(), "-30m@m")

 I still do not understand what was wrong with original search string syntax

0 Karma
Get Updates on the Splunk Community!

Join Us at the Builder Bar at .conf24 – Empowering Innovation and Collaboration

What is the Builder Bar? The Builder Bar is more than just a place; it's a hub of creativity, collaboration, ...

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...