topic How to ignore certain events from past and do not include them in the new search in Splunk Search

How to ignore certain events from past and do not include them in the new search

dearimranz — Tue, 19 Nov 2013 19:37:45 GMT

I have following data:

January 2013 (sample events)

field1:123abc field2:789xyz field3:567ghj

field1:dkd786 field2:cgu874 field3:1j7ut5

field1:i98udy field2:jfutid field3:4jfu76

February 2013 (sample events)

field1:99yekf field2:mkioie field3:34fvgh

field1:klou43 field2:ccxx45 field3:loaq56

field1:i98udy field2:jfutid field3:4jfu76 (exists in January 2013 / maybe before)

March 2013 (sample events)

field1:poph34 field2:cvt87q field3:45fgty

field1:klou43 field2:ccxx45 field3:loaq56 (exists in February 2013 / maybe before)

field1:nbty67 field2:23sxcr field3:oiu765

I have written some regexs to extract different fields and make reports out of it which works fine. However for some of the reports I have a requirement that if field1's value exists in the previous month(s) events, it should NOT show in the current month's report. The current month report should ONLY show the new fields.

Any ideas how to accomplish this. Many thanks in advance.

Re: How to ignore certain events from past and do not include them in the new search

aholzer — Tue, 19 Nov 2013 19:47:08 GMT

Try "... | dedup field1 sortby _time". It'll remove duplicates giving you the earliest occurrence of the value in the field

Re: How to ignore certain events from past and do not include them in the new search

somesoni2 — Tue, 19 Nov 2013 20:01:00 GMT

Try following. This will first get list of all the months in which a particular combination of field1, field2, field3 occurs. If count of months for a combination is more than 1, it will be excluded. Also, if there are only one month, month value should match with current month, else it will be excluded too.

...... |stats values(date_month) as months by field1, field2, field3 | where mvcount(months)=1 AND isnotnull(mvfind(months,lower(strftime(now(),"%B"))))