Splunk Search

Not eventtype

landen99
Motivator

Is it possible to create an eventtype called dns_google set as "src_ip=8.8.8.8 src_ip=4.2.2.2" and then treat it like a subsearch with a "not" like this:

mysearch NOT (eventtype=dns_google)

The goal would be to filter our 8.8.8.8 and 4.2.2.2 without putting "not" (!) into the eventtype itself so that other searches could use dns_google (for example) to match events which came from those ip addresses. The address list I have in mind is a bit larger but probably not large enough to merit a lookup table, which I still need to practice creating and using.

In this case, the task is only an example and not an important part of the question, whereas understanding if eventtype can be used with a "not" is the primary purpose here.

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Yeah, this is straightforward with eventtypes. They're just syntactic sugar for their long-form search syntax, so you can combine them with a NOT operator. To use your example, you'd define an eventtype like this in eventtypes.conf (note the OR operator!):

[google_dns]
search = src_ip=8.8.8.8 OR src_ip=4.2.2.2

Then you can do a search like this to only load Google's DNS servers:

yoursearch eventtype=dns_google

That gets translated behind the scenes into this (see the job inspector's normalizedSearch output:

yoursearch ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

Similarly, if you search for this:

yoursearch NOT eventtype=dns_google

That gets translated into this:

yoursearch NOT ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

If your eventtype is a large list of values for a single field, you're probably better off using tags as Kristian suggested. That way you can add and edit individual values without touching a huge monolithic search string. Both work in similar fashion though, as syntactic sugar for the long-form search language, and support the NOT operator.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Yeah, this is straightforward with eventtypes. They're just syntactic sugar for their long-form search syntax, so you can combine them with a NOT operator. To use your example, you'd define an eventtype like this in eventtypes.conf (note the OR operator!):

[google_dns]
search = src_ip=8.8.8.8 OR src_ip=4.2.2.2

Then you can do a search like this to only load Google's DNS servers:

yoursearch eventtype=dns_google

That gets translated behind the scenes into this (see the job inspector's normalizedSearch output:

yoursearch ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

Similarly, if you search for this:

yoursearch NOT eventtype=dns_google

That gets translated into this:

yoursearch NOT ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

If your eventtype is a large list of values for a single field, you're probably better off using tags as Kristian suggested. That way you can add and edit individual values without touching a huge monolithic search string. Both work in similar fashion though, as syntactic sugar for the long-form search language, and support the NOT operator.

martin_mueller
SplunkTrust
SplunkTrust

Ah. Entering more than one field-value pair will make that entry useless. For example, entering "host=a host=b" for tag "foo" will create a stanza in tags.conf like this:

[host=a%20host%3Db]
foo = enabled

In other words, it's looking for a host called "a host=b" - that's a valid field value (maybe not for host though), but probably not what you want.

0 Karma

landen99
Motivator

I am talking about the text field located at the Web interface-Settings-Tags-List by tag name-New-Field Value pair I agree that "sed'ing that list into a list of tags.conf stanzas" is the most approach to Tag creation.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Yeah, boolean operators are all upper caps - else you're looking for the word "not".

What "field-value box" are you talking about? It's best to post the tags.conf stanzas in question, then we are all talking about the same thing.

0 Karma

landen99
Motivator

I learned through trying both eventtype and tags in this context that it is vital that the "not" be all uppercase with parenthesis around the terms: NOT (eventtype=event1) NOT (tag=tag1)
Issue: A tag field-value combination with an * did not remove a matching domain when the NOT operator was used on it. Thus my confidence in tags has dropped. I am still curious whether multiple field value search terms within a single tag field-value box breaks the tag or not. Are those boxes just to allow for search term separation, searching, and association manipulation of tags field-value combinations?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

For batch-adding a large number of tags from a precompiled list you're better off sed'ing that list into a list of tags.conf stanzas and adding that to the config file manually. Through the UI you're going to be busy for a long time adding hundreds or thousands of tags.

0 Karma

landen99
Motivator

Perhaps you meant that tags are not well suited for large lists because you have to add each field-value pair individually, one at a time, whereas eventtypes allow for a large list to be pasted into the definition box all at once. I have just saved many pairs successfully into a single field-value pair box. Will that work properly or must each pair be placed in its own box?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

As a result, editing single values does not require touching the entire list but rather an individual stanza.

As an additional bonus, the regular Splunk Search UI allows you to graphically add new values to an existing tag.

Looking at the search inspector for a search with tags in it you can see that this:

eventSearch:    search index=_internal tag=foo

Gets translated into this (I've tagged two sourcetypes with foo):

normalizedSearch:   litsearch index=_internal ( sourcetype=splunk_web_service OR sourcetype=splunkd_access )

That's very similar to the translation for eventtypes.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

From a searching / performance POV, tags and eventtypes behave very similarly. Both get translated into their long-form SPL representation, and both support operators such as NOT.

The difference I see is in maintaining that list of IPs. With an eventtype you have one eventtype that contains a very long list of src_ip=foo OR ... entries, so to for example remove an entry you need to manually edit a large string.
On the other hand, tags are stored as individual values under a common name like this:

[src_ip=8.8.8.8]
google_dns=enabled

[src_ip=8.4.4.4]
google_dns=enabled

landen99
Motivator

For the record in support of this question/thread, the linked page on tags gives the following example use of tags grouping the same IP addresses by different criteria: tag=router tag=SF NOT (tag=Building1)

0 Karma

landen99
Motivator

I really like the idea of tags but I can't see the difference between them and eventtypes. Why are large lists better done with tags than with eventtypes? Is it because a large list in an eventtype pushes down all the other eventtypes to make it hard to find the eventtype of interest in the eventtype manager page?

0 Karma

kristian_kolb
Ultra Champion

well put.

/k

0 Karma

kristian_kolb
Ultra Champion
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...