How to search errors for last month and the current month, and find what errors are new based on the error message?

[tragiccode]

I haven't found any resource in the threads on how to do this, but what I would like to do is ask Splunk:

Get me errors for last month....then get me errors for this month. Tell me what errors "Look" new based on the error message (keeping in mind this error message is dynamic and the following 2 events should be considered the same and no considered new)

OrderNumber 2332 Failed to process
OrderNumber 8421 Failed to process

[sundareshr]

This search will get you all the errorMessages

base search earliest=-60d@d | eval when=if(_time>=relative_time(now(), "-30d@d", "This", "Last") | stats values(when) as when by errorMessage

To show errorMessages that appeared only one of the two months (either This OR Last), add this

| where mvcount(when)=1

To show errorMessages that appeared only This month but NOT Last, add this to the first search

| where isnull(mvfind(when, "Last")

To show errorMessages that appeared only Last month but NOT This add this to the first search

| where isnull(mvfind(when, "This")

[tragiccode]

I cannot try this yet but I don't really see how this would handle likeness and being able to see that 2 errors are the same if their content is slightly different

[sundareshr]

This assumes likeness is addressed in the errorMessage. To identify similar messages, you could try using the cluster command

[tragiccode]

Okay gonna need a little more help here lol. I do understand the cluster command but not quite sure how to get it to work with the above filter to identify what looks like new errors Below is what i have so far

index="log4net" AND level="ERROR" earliest=-60d@d 
 | cluster showcount=t labelonly=t 
 | table cluster_label cluster_count message
 | eval lastMonth=if(_time < relative_time(now(), "-30d@d"), 1, 0)
 | eval thisMonth=if(_time >= relative_time(now(), "-30d@d"), 1, 0)
 | stats values(lastMonth) as "last" values(thisMonth) as "this" by message
 | eval myCount = mvcount(this)
 | search this=1 AND last=0 AND myCount < 2
 | table last, this, message

[sundareshr]

Try this

index="log4net" AND level="ERROR" earliest=-60d@d 
| cluster showcount=t labelonly=t labelfield=matches
| table _time matches message
| eval when=if(_time>=relative_time(now(), "-30d@d", "This", "Last") 
| stats values(when) as when by message matches
| where isnull(mvfind(when, "Last")

[tragiccode]

Showing errors for this month but when i look at the previous month they are showing up there as well. This is not quite right :(....

index="log4net" AND level="ERROR" earliest=-60d@d
 | cluster showcount=t labelonly=t 
 | eval lastMonth=if(_time < relative_time(now(), "-30d@d"), 1, 0)
 | eval thisMonth=if(_time >= relative_time(now(), "-30d@d"), 1, 0)
 | table lastMonth thisMonth cluster_label message

The following returns results like below

lastMonth thisMonth cluster_label message
0 1 2 The order number 4184061 was not found
1 1 2 The order number 6546884 was not found
0 1 3 There was a problem with order 6546884

How do i get it to return only cluster 3. Which says this is a new cluster and the events in this cluster occurred this month and none occurred last month therefore this looks like a new error

[gokadroid]

Assuming that you already have the error messages extracted in a field called errorMessage then try this please to see if it resolves your query:

your query to return events earliest=-60d@d
| eval lastMonth=if(_time < relative_time(now(), "-30d@d"), 1, 0)
|eval thisMonth=if(_time >= relative_time(now(), "-30d@d"), 1, 0)
|stats values(lastMonth) as "last" values(thisMonth) as "this" by errorMessage
| eval myCount = mvcount(this)
| search this=1 AND last=0 AND myCount < 2
| table errorMessage

If the errorMessage is not extracted then we will have to fit in a rex line in it to extract the field errorMessage based on your exact event data.