topic Re: rare command in Splunk Search

rare command

mcm10285 — Tue, 18 Jun 2013 03:56:55 GMT

Hi, just curious how the rare command qualifies a field as rare.

Re: rare command

kristian_kolb — Tue, 18 Jun 2013 06:26:30 GMT

The least common values of a field within the timeframe. Not the rarity of the field as such.

Opposite of top

Re: rare command

MuS — Tue, 18 Jun 2013 06:59:37 GMT

addition to /K perfect answer, you can find a description for any search command in the docs 😉
see this one for rare: http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Rare

Re: rare command

mcm10285 — Wed, 19 Jun 2013 00:30:16 GMT

Thanks /K and MuS...I was really asking about the value in a field, not the field itself, thanks for clarifying that...
But let's say I'm searching user-agents being used in the environment, how does it say a result is rare? is it the number of counts it appeared in the search? If it is the count, is there a value/threshold/algorithm "rare" is looking into? That is what I would like to understand.

Re: rare command

kristian_kolb — Thu, 20 Jun 2013 07:31:16 GMT

rare will look at all the values for a given field in the search results and return a list of the least common ones. There is no magic. "How does it say that a result is rare?". Because it occurs fewer times than other values.

It's probably easier if you start playing with it to get a better understanding. It is not a complicated command. and it's well documented.

Re: rare command

mcm10285 — Tue, 25 Jun 2013 02:09:04 GMT

/K I understood your initial answer...it is comparative to those with higher values. My curiosity is the ratio by which it decides it is rare..is it 1:1000? 1:5000? etc...does it compare with the highest count? ave count? those are the lines of my question....

Re: rare command

Ayn — Tue, 25 Jun 2013 05:58:36 GMT

Like kkolb said, it's the opposite of top - "rare" values of fields are simply the ones that are more rare than the others. If you choose the 10 rarest field values, well then you will get the 10 values for that field that occurred the least in your result set. There's no hidden statistical algorithm or anything like that.

Re: rare command

kristian_kolb — Tue, 25 Jun 2013 08:20:32 GMT

To make it clear; Let's say you search for some login events in the file logins.log.

These look like this;

<time> user=bob action=login <time> user=larry action=login <time> user=bob action=logout <time> user=larry action=logout <time> user=bob action=login <time> user=frank action=login <time> user=angela action=logout <time> user=larry action=logout <time> user=bob action=logout <time> user=larry action=login <time> user=angela action=logout

A search for ...| rare limit=2 user would give you;

frank 1
angela 2

because they are the 2 least common users in the search results.

Re: rare command

mcm10285 — Tue, 25 Jun 2013 09:25:02 GMT

Ayn and /K, thanks for the detailed explanation. That clarifies it.