Splunk App for Unix and Linux: How do I prevent overlapping groups of events running a "stats count by eventtype" search?

[CREVITCH]

I am trying to chart different types of login events. I believe that the fields were extracted by the *nix app in logs from /var/log/secure.
For example, if I have 100 events, I have found that I can group them into non overlapping groups that total to 100 in the following way:

eventtype=failed_login
vendor_action=session open
vendor_action=accepted

I can create 2 new event types to cover session open and accepted.

I want to be able to do something like "stats count by eventtype" but eventtype alone generates overlapping groups of events since one event can have multiple eventtypes. Is there some way I can eliminate the unwanted event types?

[woodcock]

You can easily clone and flatten eventtype by priority like this:

... | myEventType=case(
eventtype==failed_login, "failed_login",
eventtype==something_else, "some_other_thing",
true(), null())
| stats count by myEventType

[jeffland · Accepted solution]

"Unwanted event types" is not something a machine can do for you without some hints as to which event types you are interested in. But that's ok, you just have to be aware of what behavior you actually desire, and aware of what event types are.
Event types are based on searches for text, so all events that contain either "failed login" or "login_failure" (or whatever else is in the event type definition) will have the event type "failed_login". As you already noted, an event can contain text that qualifies for more than one event type. This means you will have to decide which of these event types you want to count, as both are equally valid to start with.

Suppose you have events with event types "login_attempt", "failed_login", "known_username" and "successful_login", then I would imagine you are most interested in a failed login. Therefore, you could want to count this event as a "failed_login" regardless of whether this event also has the event type "login_attempt". However, it might also be interesting to count failed logins with and without "known_username", in order to distinguish between failed logins for existing users and failed logins for unknown users - see what I'm getting at? It's not trivial which of these event types is the most important one, and sometimes you may want to incorporate information from more than one event type into two before counting them.

One way around this is to create a new field, and in that field have the most important event type of any one event or a combination of these events - in case of the event types above, you could have the following eval help you out with that:

... | eval type=case(eventtype="failed_login" AND eventtype="known_username", "Known username, failed login", eventtype="failed_login", "Unknown username, failed login", ...) | stats count by type

Since the case statement can only ever result in one entry, you will not get overlapping numbers here. How the events are assigned to one of your cases is up to you however.