How to combine my 3 searches into one to create an alert?

[aramakrishnan]

I'm trying to set up an alert for the time taken for a process, which I was previously calculating using 3 separate searches. I'm looking to combine them into one so I can directly set up an alert.

(1) The background of the process is that every serial number in the sourcetype has a number of segment Ids, starting from 0. While all serial numbers start with segment =0, the largest segment number is different for all devices. So the way I compute largest segment ID is as follows:

sourcetype="algorithmjoblog" serialNumber="NTEST1234B" | stats max(segmentId) as lastSeg

(2) Next, I calculate duration for segmentId=0, processing phase=mapstart (i.e., the segment has started processing) and segmentId=max segment, processing phase=mapEnd (i.e., the segment processing has completed).

sourcetype="algorithmjoblog" serialNumber="NTEST1234B" | transaction startswith=(segmentId=0 processingPhase=mapStart) endswith=(segmentId=165 processingPhase=mapEnd)| table duration

(3) Finally, I calculate the time taken for the final stage of the process, which is integrationStart and integrationEnd, which all serialNumbers go through.

sourcetype="algorithmjoblog" serialNumber="NTEST1234B" | transaction startswith=(phase=integrationStart) endswith=(phase=integrationEnd)| table duration

I'm interested in setting up an alert for the sum of duration of (2) and (3), i.e., segment processing time + integration time, such that I get an alert when the time exceeds a value, say 10 mins. Also, I would like the alert to encompass all serialnumbers in a search, not just one specific serial number at a time.

I have an updated search, but the problem is that it's only using integration times for the first serialNumber, instead of extracting it from every serial number.

Here's what i ran:

sourcetype=algorithmjoblog serialNumber="NTEST*" |eval p_{processingPhase}=_time |stats first(p_*) as * by serialNumber | eval duration1=mapEnd-mapStart | table  duration1| join serialNumber[search sourcetype=algorithmjoblog serialNumber="NTEST*" | transaction startswith=(phase=integrationStart) endswith=(phase=integrationEnd) by serialNumber|eval duration2=duration| table duration2]

This is what my table looks like:

duration 1      duration2
123.45          101.45
233.34          101.45
3232.234        101.45
23.23           101.45
234.33          101.45

...where 101.45 is the integration time (duration2) for the first serial number (i.e. with duration 1 as 123.45).

Any inputs on how I could correct my search?

Thank you so much in advance for any help!

[stephane_cyrill]

Hi try this to merge your searches,

sourcetype="algorithmjoblog" serialNumber="NTEST*" | transaction startswith=(segmentId=0 processingPhase=mapStart) endswith=(segmentId=165 processingPhase=mapEnd)|eval duration1=duration| table duration1 [search sourcetype="algorithmjoblog" serialNumber="NTEST*" | transaction startswith=(phase=integrationStart) endswith=(phase=integrationEnd)|eval duration2=duration| table duration2] |eval duration3= duration1 + duration2| eval Test_duration=if(duration3>10, "the time exceeds","No")|table Test_duration

You can set up your alert base on the value of the field Test_duration

[aramakrishnan]

I have an updated query that fixes the problem in the comment above, but I now have a new problem where it's only using integration times for the first serialNumber, instead of extracting it from every serial number.

Here's what i ran:

sourcetype=algorithmjoblog serialNumber="NTEST*" |eval p_{processingPhase}=_time |stats first(p_*) as * by serialNumber | eval duration1=mapEnd-mapStart | table  duration1| join serialNumber[search sourcetype=algorithmjoblog serialNumber="NTEST*" | transaction startswith=(phase=integrationStart) endswith=(phase=integrationEnd) by serialNumber|eval duration2=duration| table duration2]

This is what my table looks like

duration 1            duration2
123.45            101.45
233.34            101.45
3232.234            101.45
23.23            101.45
234.33            101.45

...where 101.45 is the integration time (duration2) for the first serial number (i.e. with duration 1 as 123.45).

Any inputs on how I could correct my search?

[aramakrishnan]

Thanks @stephane_cyrille, I still have couple issues. (1) The maximum segment ID is not always 165 as in the query above (it varies per serialNumber), and earlier I was using a separate query for that (sourcetype="algorithmjoblog" serialNumber="NTEST1234B" | stats max(segmentId) as lastSeg). I changed this to sourcetype="algorithmjoblog" serialNumber="NTEST1234B" | stats max(segmentId) as lastSeg by serialNumber, but it doesn't work when I include it in the combined query.

How do I set it up such that it pulls out the maximum segment ID for every device and returns that as a value that I can use in the "endswith" section of the query? If I say endswith=(segmentId=lastSeg processingPhase=mapEnd), it doesn't work.

(2) Also, when I run the query it says "Error in 'table' command: Invalid argumentL 'duration2 =100.655". What do you think may be the issue?

Thanks a lot for your help!