audit1.csv was the file that I uploaded on Day1 and audit2.csv was the file that I uploaded on Day2. Could it be that because Splunk search in both files that why is Robert appeared in Day 1 and also Day 2?  Very respectfully,  Long    ... View more

Hi ITWhisperer, Thanks for always responding to my questions. It did not work. On day 2, Robert still appears. I believe your queries are supposed to overwrite the alerts.csv on Day1 and on Day2 there should be James, David, Richard, and William appearing on the alerts.csv file. However, this query lookups everyone and keep them in the alerts.csv file on day2.  On Day3, because the query also finds Robert on Day2 (which was supposed to be deleted), therefore, it is not sending another alert. In this case, Robert's case is just like James who appears for 3-day straight because Splunk does not overwrite the alerts.csv and delete Robert on Day2. Please advise and please let me know if I am wrong! Very respectfully,  Long      ... View more

I wonder if it is better for me to try working this on kv store instead? Thanks ... View more

Hi ITWhisperer,  Thanks for your prompt response. I implemented your logic on Splunk 8.2.1. and Robert still pops up on Day2 even though he is not flagged on Day2 csv file.  Also, no alert was sent to Robert on Day3 as well even though he is flagged on Day3.  Additionally, I implemented your logic on paid Splunk virtual environment and another dataset and I've got the same result.    Here is how I set up condition.csv file Could you please advise? Very respectfully, Long    ... View more

Hi  ITWhisperer,   Thanks for all your responses. I am testing your logic on the paid Splunk virtual environment and Splunk still does not send another alert for the person on day3 when he is flagged and is sent an alert on Day1, disappear on Day2, and flagged again on Day3. I wonder if it works for you?  Very respectfully, Long  ... View more

Hi ITWhisperer, Thanks for your response, sir. Yes, I also noticed that Robert appears in Day2. Here is the content of the condition.csv file.  Very respectfully, Long  ... View more

Hi ITWhisperer, Thanks for your response, sir. Yes, I also noticed that Robert appears in Day2. Here is the content of the condition.csv file.  Very respectfully, Long  ... View more

Hi ITWhisper,   It is not fixed and the error still persists. James , Robert John, Michael has "COUNTRY=CAN" and Splunk sends them alert on Day1.    James, William, David, and Richard are flagged on Day2 and Splunk only sends alert for William, David, and Richard. Splunk does not send an alert for James because his case is continuity.  That is perfect.  On Day3, James, Robert, Thomas, and Charles are flagged. Splunk did not send an alert for James (This is good because his case is continuity). Splunk sends an alert for Charles and Thomas (This is also good). However, Splunk does not send an alert for Robert (which is not good because Robert is flagged on Day1, he is not flagged on Day2 but he is flagged again on Day3. Hence, Splunk is supposed to send an alert to Robert). Could you please help me fixing Robert's condition?  Also, if I need to split this search, for example, one search is for continuity under 7 days and another search is for 24-hour break, just like Robert here, and combine both search somewhere? Please advise! Very respectfully, Long    ... View more

Hi ITWhsiper, Thanks a lot for your help.  There is one missing piece in this.  Robert appears on Day1 and spunk sends an alert "Y".  He disappears on Day2  He appears again on Day3 but Spunk does not send another alert.  What query do I need to add here so that Splunk will send an alert for Robert on Day 3? You used to give me this    | eval _send_alert=if(isnull(Alert_sent_date) OR Alert_date - Alert_sent_date > 60 * 2, "Y",null())   I wonder how do I integrate it into this to make it work. Could you please advise? | inputlookup audit.csv | lookup condition.csv Department Name Role email employee_id | where watch="Y" | fields email employee_id | eval Alert_date=relative_time(now(),"@m") | append [| inputlookup alerts.csv | where relative_time(now(),"@m")-Alert_sent_date < 60*7 OR employee_id = 0] | stats max(*) as * by email employee_id | eval _send_alert=if(isnull(Alert_sent_date), "Y",null()) | eval Alert_sent_date=if(_send_alert="Y", Alert_date,Alert_sent_date) | eval Alert_updated_date=Alert_date | rename Alert_date as _Alert_date | outputlookup alerts.csv | where isnotnull(_Alert_date) | eval send_alert=if(_send_alert="Y",_send_alert,"N") | fieldformat Alert_sent_date=strftime(Alert_sent_date,"%H:%M:%S") | fieldformat Alert_updated_date=strftime(Alert_updated_date,"%H:%M:%S") Very respectfully, Long      ... View more

HI ITWhisperer, Thank you for taking your time making the table. That is right, sir, you are correct, these are the situations that I hope to implement in Splunk. I wonder if there are ways that I could write queries based on these conditions in the search?    Very respectfully, Long  ... View more

Hi ITWhisper, Thank you for your response. It has been a really great help! However, I think I am missing a logic where Splunk flags an employee continuously. When Splunk flags an employee continuously in less than or equal to 7 days I want Splunk to send an alert for only once time, which is at the beginning. In the below example, Splunk flags James Julie continuously for seven days. And it sends the alert when (Alert_update_date - Alert_Sent_Date> 24*60*60) is true. I wonder if there is a way that I could write a query to make an exception when Splunk flags Jame Julie continuously without a break, I want Splunk to send a flag for only once. Also, this continuity is only for 7 days and Splunk will send another alert on Day8.In addition, I want Splunk to keep the original condition that it will send an alert when there is a break in between. Could you please advise? Have a great day! Very respectfully, Long    ... View more

@ITWhisperer wrote: In condition.csv you have John's role as "Staff" whereas in your audit.csv you have it as "staff" - these do not match, hence zero rows from the where clause. Hi ITWhisper, Again thanks for your response, sir. I successfully extracted the employees that meet the condition.csv.  I have several other questions and I hope you could help me out.  1. I set the alert to throttle and have it sent for every 5 minutes. However,  the Alert_updated_date and send_alert have not been populated with new data. How do I change this?     2. Also, if Splunk identifies the same employee as a flag continuously for the next seven days, I do not want Splunk to send an alert  everyday for the same employee. I want Splunk to send an alert for only the first time it flags the employee. What should I look into this? 3. In the case that Splunk flag an employee on the first day and Splunk does not catches the same employee again for the next 6 days, I would want Splunk to delete the employee information from its database. How could I achieve that? So ideally, I am uploading a cvs file everyday onto Splunk and have it flagged the employees that meet the criteria in condition.csv.  If employee A is flagged on the Day#1and disappear for the next 6 days, I would want Splunk to delete his profile from its database. Also,  if Splunk flagged employee A on Day#1 Day#2......Day#7 continuously, I want Splunk to send the alert for only Day #1 However, if there is a break when Splunk flagged employee A for example it catches employee A on Day#1 but NOT Day #2 but flag again on Day#3. I would want Splunk to send an alert on Day#1 and Day#3.  Since there are thousand of columns and rows in csv files, I wonder if there is a way to make the search more efficient since the time frame is 7 days. I wonder if there is a way that I could have Splunk deleted the flagged employees that is older than 7 days.  Please advise! Very respectfully, Long    ... View more

Hi ITWhisperer, Again, greatly appreciated for your response. As I went through your instruction, everything went smoothly through #2 line.  However, I've got nothing in return for the third command    As shown in the screenshot below,  inputlookup condition.csv has all the criteria that should be called in the above search. Could you please advise?   very respectfully, Long    ... View more

Hi ITWhisperer, Thank you for your prompt response. In regard to your search commands, I wonder if you could help me explaining how it works. #1 | eval send_alert1=if(isnull(Alert_sent_date) OR Alert_date - Alert_sent_date > 24 * 60 * 60 , "Y" , null()) My thought is it create send_alert1 If 1# the Alert_sent_date column is blank Or 2# if the Alert_date – Alert_sent_date is greater than 24*60*60 (in second) Else: Return null #2 | eval Alert_sent_date=if(send_alert1="Y" , Alert_date,Alert_sent_date) This command creates a column for Alert_set_date if send_Alert1 = “Y” and have Alert_sent_date = Alert_date.  #3  | outputlookup append=true alerts.csv This command is updating the alerts.csv file with the append command #4 | eval send_alert=if(send_alert1="Y", send_alert1, "N") This command is creating the send_alert column and set it to send_alert 1 if sent_alert1 is “Y” else set it to “N” These are my understanding in regard to your search commands. Please let me know if my understanding is aligning with yours or please explain the the difference. Very respectfully, Long      ... View more

Hi ITWhisper, Thank you for your response. It was very helpful in taking me to the next step. I integrated your logic into my Splunk and there are a few more things that I would like to ask.  1. Why is the Alert_sent_date is populated to 7/15/2021 00:00:00 when I have not enabled and set the triggered  alert to the current date at 00:00:00 yet?  2. How do I connect this search to alert triggered? Could I just write a query alert into the search or should I just go to Save As > Alert? Which method is better and cleaner since I will have a lot more queries and logic to add to this search?  Very respectfully, Long      ... View more

Hi ITWhisperer, Thank you for your kind response.  I uploaded this csv file onto Splunk.  my goal is to use KV store search to catch a flag if the condition below is met and it will come out as below Additionally, I want Splunk to send an alert email based on the flag in this format  1. So, for example, on day 1, a cvs file will be uploaded onto Splunk and Splunk will run to see if there is a department = "Business and Economics" and Role = staff. If Splunk catches this flag, it means that there is a policy violation and Splunk will  send an alert to the admin immediately for a remediation. The report that is send to the admin will have the email and employee_id. Also, it will have the date that the alert_sent_date for the employee who violates the policy and also the date_updated.  2. On the second day, when the second csv file is uploaded onto Splunk and if the same employee is caught violating the policy, there is no need to send an alert to the admin. However, there needs to be an update onto Splunk like this 3. On day 3, when the csv file is uploaded onto Splunk and if the same employee violating the policy, Splunk will send an alert to the admin and the report will need to look like this: I know that I need to use eval like this : source = “Splunk Questions.csv” Role = Staff Department = “Business and Economics” | eval Alert_Sent_Date =  | eval Alert_Updated_Date = However, I do not know how to pick up the Alert_Sent_Date and Date_Updated from Splunk and add it to the alert report and also how to update it on the daily basis when I upload the csv file to Splunk everyday.  Also,  it is only running for seven days. If the same employee has not violated the policy for the next seven days, I also want to have Splunk deleted that employee data from the database. This would be for later discussion. Anything that I could look into? Could you please advise?  Very respectfully, Long   ... View more