Multiple error stements with retry

[markthompson]

Hello,
Well we have a job that runs and produces log files that runs and if it fails, it retries up to 3x.

How would we get splunk to take the latest value for that day of the log file so we can detect whether or not it has failed overall or not.

Thanks

[markthompson · Accepted solution]

I have solved this using the eventstats command, example below:
index="{INDEX}" host={HOST} |
transaction startswith="{STARTSWITH}" endswith="{ENDSWITH}" |
eval StartTime=_time |
eventstats max(StartTime) as max |
where max=StartTime

This will select the latest event only.

[martin_mueller]

Try this:

index=foo sourcetype=bar "the run has been completed successfully" | timechart span=1d count | where count=0

That should give you days without a successful run.

[markthompson]

Hi Martin, thats not particularly what we're looking for, we are using rangemaps to display traffic lights of the current status, and if it's warning but is succeeding after retry it should go amber, if its failing altogether, red, and if successful then green.

[martin_mueller]

What does a failure look like in the logs? What does a successful run look like?

[markthompson]

Hey Martin,
it will be {Timestamp}: Error - {Error type}
success will be something like The run has been completed successfully.