search correlated events

[gudavasr]

I have a log like this: (this is from search...| transaction command)

[LOG|DEBUG|28 May 2015 15:42:28,722|com.XL.Source|Thread-19|-]
Deal - 123456 (New,PickedUp,3) notification received from Source
[END]
[LOG|DEBUG|28 May 2015 15:42:40,530|com.XL.Source|EngineThread1|-]
Deal - 123456 (New,PickedUp,3) - Publishing SAVED ack for the deal ...
[END]

This is the search:

source="**.log" ("(New*,Picked*" "notification received from Source" OR "SAVED ack")  
 | rex field=_raw "Deal -\s*(?<MW_Deal_Id>\d+) \("   
| transaction MW_Deal_Id   
| rex field=_raw "(?<MWDealBookedByUserTime>\d+ \w+ \d+ \d+:\d+:\d+,\d+)\|com.XL.Source\|Thread-*"   
| rex field=_raw "(?<MWDealreceivedbyXLTime>\d+ \w+ \d+ \d+:\d+:\d+,\d+)\|com.XL.Source\|EngineThread"   
| eval boo=strptime(MWDealBookedByUserTime, "%d %b %Y %H:%M:%S,%N")    
| eval foo=strptime(MWDealreceivedbyXLTime, "%d %b %Y %H:%M:%S,%N")    
| eval deal_duration_time_in_min = (foo - boo)/60   
| eval dealDurationinMin=if(deal_duration_time_in_min>0, deal_duration_time_in_min,  "Deal Still not saved in XL")

This one works fine as report, but as an alert that runs every 5 mins, it is sending false alerts.
Can you please help how can avoid false alert ?

i tried "grep -A" kind of search when this line is found
[LOG|DEBUG|28 May 2015 15:42:28,722|com.XL.Source|Thread-19|-]

but was not successful. hence I end up using above search.

[woodcock]

Try this (not using transaction) and see if it works better:

source="**.log" ("(New*,Picked*" "notification received from Source" OR "SAVED ack")
| rex field=_raw "Deal -s*(?<MW_Deal_Id>d+) (" 
| stats values(_raw) AS mutli_raw BY MW_Deal_Id 
| rex field=mutli_raw"(?<MWDealBookedByUserTime>d+ w+ d+ d+:d+:d+,d+)|com.XL.Source|Thread-*" 
| rex field=mutli_raw"(?<MWDealreceivedbyXLTime>d+ w+ d+ d+:d+:d+,d+)|com.XL.Source|EngineThread"
| eval boo=strptime(MWDealBookedByUserTime, "%d %b %Y %H:%M:%S,%N") 
| eval foo=strptime(MWDealreceivedbyXLTime, "%d %b %Y %H:%M:%S,%N") 
| eval deal_duration_time_in_min = (foo - boo)/60 
| eval dealDurationinMin=if(deal_duration_time_in_min>0, deal_duration_time_in_min, "Deal Still not saved in XL")

[gudavasr]

for some reason it's not creating these fields:

MWDealBookedByUserTime and MWDealreceivedbyXLTime
and because of that it's not working.

[woodcock]

OK, try this:

source="**.log" ("(New*,Picked*" "notification received from Source" OR "SAVED ack")
| rex "Deal -s*(?<MW_Deal_Id>d+) ("
| rex "(?<MWDealBookedByUserTime>d+ w+ d+ d+:d+:d+,d+)|com.XL.Source|Thread-*"
| rex "(?<MWDealreceivedbyXLTime>d+ w+ d+ d+:d+:d+,d+)|com.XL.Source|EngineThread"
| stats earliest(MWDealBookedByUserTime) AS MWDealBookedByUserTime
             latest(MWDealreceivedbyXLTime) AS MWDealreceivedbyXLTime BY MW_Deal_Id
| eval boo=strptime(MWDealBookedByUserTime, "%d %b %Y %H:%M:%S,%N")
| eval foo=strptime(MWDealreceivedbyXLTime, "%d %b %Y %H:%M:%S,%N")
| eval deal_duration_time_in_min = (foo - boo)/60
| eval dealDurationinMin=if(deal_duration_time_in_min>0, deal_duration_time_in_min, "Deal Still not saved in XL")

[gudavasr]

little better..I will try this and update this question.
Thank You.