Not eventtype

[landen99]

Is it possible to create an eventtype called dns_google set as "src_ip=8.8.8.8 src_ip=4.2.2.2" and then treat it like a subsearch with a "not" like this:

mysearch NOT (eventtype=dns_google)

The goal would be to filter our 8.8.8.8 and 4.2.2.2 without putting "not" (!) into the eventtype itself so that other searches could use dns_google (for example) to match events which came from those ip addresses. The address list I have in mind is a bit larger but probably not large enough to merit a lookup table, which I still need to practice creating and using.

In this case, the task is only an example and not an important part of the question, whereas understanding if eventtype can be used with a "not" is the primary purpose here.

[martin_mueller · Accepted solution]

Yeah, this is straightforward with eventtypes. They're just syntactic sugar for their long-form search syntax, so you can combine them with a NOT operator. To use your example, you'd define an eventtype like this in eventtypes.conf (note the OR operator!):

[google_dns]
search = src_ip=8.8.8.8 OR src_ip=4.2.2.2

Then you can do a search like this to only load Google's DNS servers:

yoursearch eventtype=dns_google

That gets translated behind the scenes into this (see the job inspector's normalizedSearch output:

yoursearch ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

Similarly, if you search for this:

yoursearch NOT eventtype=dns_google

That gets translated into this:

yoursearch NOT ( ( src_ip=8.8.8.8 OR src_ip=4.2.2.2 ) )

If your eventtype is a large list of values for a single field, you're probably better off using tags as Kristian suggested. That way you can add and edit individual values without touching a huge monolithic search string. Both work in similar fashion though, as syntactic sugar for the long-form search language, and support the NOT operator.

[martin_mueller]

Ah. Entering more than one field-value pair will make that entry useless. For example, entering "host=a host=b" for tag "foo" will create a stanza in tags.conf like this:

[host=a%20host%3Db]
foo = enabled

In other words, it's looking for a host called "a host=b" - that's a valid field value (maybe not for host though), but probably not what you want.

[landen99]

I am talking about the text field located at the Web interface-Settings-Tags-List by tag name-New-Field Value pair I agree that "sed'ing that list into a list of tags.conf stanzas" is the most approach to Tag creation.

[martin_mueller]

Yeah, boolean operators are all upper caps - else you're looking for the word "not".

What "field-value box" are you talking about? It's best to post the tags.conf stanzas in question, then we are all talking about the same thing.

[landen99]

I learned through trying both eventtype and tags in this context that it is vital that the "not" be all uppercase with parenthesis around the terms: NOT (eventtype=event1) NOT (tag=tag1)
Issue: A tag field-value combination with an * did not remove a matching domain when the NOT operator was used on it. Thus my confidence in tags has dropped. I am still curious whether multiple field value search terms within a single tag field-value box breaks the tag or not. Are those boxes just to allow for search term separation, searching, and association manipulation of tags field-value combinations?

[martin_mueller]

For batch-adding a large number of tags from a precompiled list you're better off sed'ing that list into a list of tags.conf stanzas and adding that to the config file manually. Through the UI you're going to be busy for a long time adding hundreds or thousands of tags.

[landen99]

Perhaps you meant that tags are not well suited for large lists because you have to add each field-value pair individually, one at a time, whereas eventtypes allow for a large list to be pasted into the definition box all at once. I have just saved many pairs successfully into a single field-value pair box. Will that work properly or must each pair be placed in its own box?

[martin_mueller]

As a result, editing single values does not require touching the entire list but rather an individual stanza.

As an additional bonus, the regular Splunk Search UI allows you to graphically add new values to an existing tag.

Looking at the search inspector for a search with tags in it you can see that this:

eventSearch:    search index=_internal tag=foo

Gets translated into this (I've tagged two sourcetypes with foo):

normalizedSearch:   litsearch index=_internal ( sourcetype=splunk_web_service OR sourcetype=splunkd_access )

That's very similar to the translation for eventtypes.

[martin_mueller]

From a searching / performance POV, tags and eventtypes behave very similarly. Both get translated into their long-form SPL representation, and both support operators such as NOT.

The difference I see is in maintaining that list of IPs. With an eventtype you have one eventtype that contains a very long list of src_ip=foo OR ... entries, so to for example remove an entry you need to manually edit a large string.
On the other hand, tags are stored as individual values under a common name like this:

[src_ip=8.8.8.8]
google_dns=enabled

[src_ip=8.4.4.4]
google_dns=enabled

[landen99]

For the record in support of this question/thread, the linked page on tags gives the following example use of tags grouping the same IP addresses by different criteria: tag=router tag=SF NOT (tag=Building1)

[landen99]

I really like the idea of tags but I can't see the difference between them and eventtypes. Why are large lists better done with tags than with eventtypes? Is it because a large list in an eventtype pushes down all the other eventtypes to make it hard to find the eventtype of interest in the eventtype manager page?

[kristian_kolb]

well put.

/k

[kristian_kolb]

Have you looked at tagging those key/value pairs?

http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Abouttagsandaliases