Transforming Search Output

[jason_hotchkiss]

base search
| fields _time host pdfpath status
| stats values(pdfpath) as pdfpath values(host) as host by _time status
| table _time host status pdfpath

Example Log

_time host status pdfpath
2021-09-08 08:00:00.359 hostA processing /20210907/xxxx_live.3.21.cv.1866.13428730.1
2021-09-08 08:00:00.458 hostB processing /20180821/xxxx_live.1.18.cr.403.19409265.0
2021-09-08 08:00:00.462 hostB processing /20180821/xxxx_live.1.18.cr.403.19409265.0
2021-09-08 08:00:00.473 hostA finished /20210907/xxxx_live.3.21.cv.1866.13428730.1
2021-09-08 08:00:00.477 hostC processing /tmp/HL_end_state379145533207037128.pdf
2021-09-08 08:00:00.500 hostC finished /tmp/HL_end_state379145533207037128.pdf

I am looking for a way to trigger an alert when a host does not finish processing a pdfpath. Using the example above,
hostB is having trouble processing it's pdfpath, as there is no correlating "finished" status as there is for hostA and hostC.

The output could be just the earliest and latest time the file was processed and a count of the attempts to process the file. It would be cool also to have a new status called "failed" to easily count the number of failures. Output example:

earliest_time latest_time host number_of_attempts pdfpath status
2021-09-08 08:00:00.458 2021-09-08 08:00:00.462 hostB 2 /20180821/xxxx_live.1.18.cr.403.19409265.0 failed

I'm looking for suggestions on how I could do this. Thank you.

[ITWhisperer · Accepted solution]

| stats earliest(_time) as earliest_time latest(_time) as latest_time latest(status) as status count by host pdfpath
| eval count=if(status="finished",count-1,count)
| eval status=if(status="processing","failed",status)

You could set your alert with the addition of where status="failed" and see if any results are returned

[jason_hotchkiss]

Thank you for assistance.  This solves my use case.