Im currently overhauling the search architecture and am looking to classify my data into types, some of which will have subtypes. Upon investigation, it seems that I could create automatic lookups to classify most of my types/subtypes, except for classifying based on when field!=value (I am unaware of a way to lookup something that doesn't exist). I could also use eventtypes to build all of the auto classification, which seem to be more flexible in terms of defining the data types (can use full search terms, not just matching key=value pairs).
Is it more efficient performance-wise to use lookup tables over eventtypes?
It seems that the lookup tables apply to only the sourcetype specified, while eventtypes would have to check every event regardless of sourcetype to see if there is a match. If I define an eventtype with some search terms and a sourcetype (e.g. sourcetype=thislog foo=value ), will Splunk only try to match assign that eventtype to the mentioned sourcetype? I read somewhere that there is at least SOME optimization for eventtypes, although it was not clearly documented or explained.
Also, what about using a calculated field to add classification data? how does that compare to eventtypes or lookups. My searches typically return around 1 million events, with occasional searches returning maybe 300 million events.
... View more