We have different levels of data flow coordinated by a set of saved searches. We divide them into three tiers where the data after 3rd tier is displayed on the dashboard. Everyday for 30-40 minutes, this data flow is interrupted. The internal logs show nothing out of the ordinary and data flow is stuck at tier 2 saved searches. There are no skipped saved searches as well during this time and the server health is at top notch as well. The trend of time range of the issue increases daily i.e. first day issue happened between 4-4:45, next day 5-5:45 and so on. Yesterday this happened between 10PM -10:45PM IST. There is no network connectivity issue as well as data is coming in tier 1 index. Please check the attached image. Thanks in advance.
... View more
I made an alert which sends out mail to the respective teams whenever a high priority task has not been updated for more than an hour. The query is as follows:-
index="abc" INC* main_metric,
state="New" OR state="In Progress" OR state="Awaiting Third Party" OR state="Pending" priority = "1 - Critical" OR priority = "2 - High"
| rex field=_raw "main_metric=\"(?<main_metric>\S+\s\d+\:\d+\:\d+)\""
| dedup main_metric
| dedup number
| eval main_metric = upper(main_metric)
| lookup lookup_inactivity_alert_distribution_list.csv assignment_group OUTPUT "Email_To" "Email_Cc" "Email_Bcc" "Enabled"
| fillnull value=0
| search number != 0 AND Enabled = "Y" AND main_metric != 0
| eval end=strptime(main_metric, "%Y-%m-%d %H:%M:%S.%N")
| eval start=now()
| eval diff = start - end
| lookup lookup_frequency_impact.csv impact output "Frequency1" "Frequency2" "Frequency3" "Frequency4"
| eval freqdiff1 = Frequency1 + 600
| eval freqdiff2 = Frequency2 + 600
| eval freqdiff3 = Frequency3 + 600
| eval freqdiff4 = Frequency4 + 600
|eval result = case('caller_id' = "SCOM System" AND
'diff' >= 'Frequency3' AND
'diff' <= 'freqdiff3',"outcome1",'caller_id' != "SCOM System" AND 'diff' >= 'Frequency1' AND
'diff' <= 'freqdiff1',"ouitcome2",'caller_id' != "SCOM System" AND 'diff' >= 'Frequency2' AND
'diff' <= 'freqdiff2',"outcome3",'caller_id' != "SCOM System" AND 'diff' >= 'Frequency3' AND
'diff' <= 'freqdiff3',"outcome4",'caller_id' != "SCOM System" AND 'diff' >= 'Frequency4' AND
'diff' <= 'freqdiff4',"outcome5",1==1,"no outcome")
| search result="outcome1" OR result="outcome2" OR result="outcome3" OR result="outcome4" OR result="outcome5" AND state!="Closed"
| table main_metric priority caller_id result assignment_group u_updated_on "Email_To" Email_Cc Email_Bcc number start end diff Frequency1 Frequency2 Frequency3 Frequency4
| map alert_main_metric_mail assignment_group="$assignment_group$" to="$Email_To$" cc="$Email_cc$" bcc="$Email_Bcc$"
The second lookup handles the frequency with which the alert emails are to be sent with respect to the priority of the ticket. Now the problem that I am having is that if the ticket or task is closed within half an hour of it being created, the alert is still generated. Even if the ticket is de-escalated, the alert is still being received. I tried many modifications in the code but nothing seems to work. Could you all help me with this bug?
P.S.: The map commands connect just the saved search which sends out the emails with the appropriate subject and description.
... View more
Hi, actually there wasn't an issue with the lookup, there was another file with which the name is matched with this lookup (defined in the savedsearch at the 1st line) and that name was inserted wrong( a "space error") .
Thank you for your response. I will definitely try this method as well as it looks more organised.
... View more
I have the following command:-
| savedsearch issue_with_lookup team="$token$" team_from_roster="$token$" team_roster_count="$token$"
| eval team="$token$"
| eval current_owner = if(current_owner = "","NA", current_owner)
| eval ID = current_owner
| where current_owner != "NA"
| lookup example.csv name as current_owner OUTPUT sys_id as user_sys_id
| table ticket_number, system_id, current_owner, assigneeID, team, reassignment, user_sys_id
| rename ticket_number as ticketName, system_id as ticketID, current_owner as assigneeName, reassignment as reassignment_flag
| search NOT
[search index=abc earliest=-6m latest=now
| dedup ticketName | table ticketName ]
Now the issue is when I run this query, all the fields occur except user_sys_id whose value is definitely present in the lookup and should reflect but is not. This is a scheduled query which runs every 6 mins.
... View more