How to exclude certain wildcard matches from the search

[franjo]

I need to search for *exception in our logs (e.g. "NullPointerException") but want to exclude certain matches (e.g. "DefaultException"). If my search is

*exception NOT DefaultException

then it works fine, except for the cases where I have both "NullPointerException" and "DefaultException" values in the text.
How do I create a search that will ignore records with "DefaultException" only, but not ignore them if there are other "*exception" present?

Here is an example: I want to ignore line1, but get (in search results) lines 2 and 3. With my search above I'm only getting line3:

1. line1: some DefaultException happened
2. line2: some NullPointerException happened together with DefaultException
3. line3: another NullPointerException happened

Thanks!
Franjo.

[morethanyell · Accepted solution]

Say you have these raw logs (as you mentioned above)

| makeresults
| eval sample_raw = "some DefaultException happened,some NullPointerException happened together with DefaultException,another NullPointerException happened"
| makemv delim="," sample_raw
| mvexpand sample_raw

The important thing to consider here is to count the occurence of "Exception" in your raw log. In order to do that, we need to capture the exceptions as multivalue field like this (or by adding this line of rex)

| rex field=sample_raw max_match=0 "(?<exception_type>[\w]+)Exception"

As a result, it will create an MV field containing all the Exceptions like this:

From here, you can just easily filter out the ones you don't like using the | where command:

| where mvcount(exception_type) > 1 OR exception_type != "Default"

I think this is just one approach. There might be better ways to do it.

[franjo]

mvCount > 1 was not always good, but if I use this then I get what I want:

where mvcount(mvfilter(exception_type != "Default")) > 0

[morethanyell]

That is a lot better

[adonio]

your search .... NOT "DefaultException" ....

[franjo]

how is this different from " *exception NOT DefaultException " that I already mentioned in the question?

[adonio]

looks like i misunderstood your question, can you elaborate on the requirement: "How do I create a search that will ignore records with "DefaultException" only, but not ignore them if there are other "*exception" present?"
can you provide some sample data?

[franjo]

Sure, here are 3 examples. I want to ignore line1, but get (in search results) lines 2 and 3. With my initial query (and your proposal) I'm only getting line3:

1. line1: some DefaultException happened
   
2. line2: some NullPointerException happened together with DefaultException
3. line3: another NullPointerException happened

[adonio]

can you provide a larger masked data set? also can you provide the final result you are interested in with the format? chart / table / etc ...

[rupesh26]

do you want to ignore events where both "NullPointerException" and "DefaultException" values present as well ?

[franjo]

No, I want to always see (get in results) my "NullPointerException". I just don't want to see the entries where only "DefaultExceptions" is present (and no other one).

[rupesh26]

when you do a wildcard search you are searching the entire event, so it will ignore even this event

some NullPointerException happened together with DefaultException

The best way to do is use field extraction and extract NullPointerException to a field and add that field to your search