topic Re: How to create a list of fields in a table with stats displayed on the Selected Fields sidebar, including rare 10 values, their counts, and percentages? in Splunk Search

How to create a list of fields in a table with stats displayed on the Selected Fields sidebar, including rare 10 values, their counts, and percentages?

landen99 — Wed, 09 Sep 2015 14:27:45 GMT

I want to take a list of fields and show the stats displayed on the Selected fields sidebar in a table.

When we do a search, on the left side there is a Selected Fields section. When we click on a field, we see stats on that field, including:
Percent of events overall
Top 10 values and their counts and percents

I would like to put these into a table for each field specified (not just top), and other stats too.

I am also interested in other stats for each field:
Rare 10 values and their counts and percents
Values which deviate from the standard pattern

Table:

Type Field Value Count %
All user ALL 99 99%
Top 1 user abc 12 12%
Top 2 ...
Bottom 1 user xyz 13 13%
All dest ALL 98 98%
...

Maybe it will involve append using top and rare, or perhaps fieldsummary? but is it possible to use eventstats? like:

| table field1 field2 |  eventstats count percent by *

fieldsummary has no percent and the values are not well formatted. Plus, there are a lot of extra fields there as well. Perhaps I need to manually format it or are there functions which assist with the formatting and rendering of the data? It also does not seem to show rare values.

Re: How to create a list of fields in a table with stats displayed on the Selected Fields sidebar, including rare 10 values, their counts, and percentages?

landen99 — Wed, 09 Sep 2015 14:40:50 GMT

Maybe:

| anomalousvalue action=filter field1 field2

to find the anomalous values but it does not compute counts and percentages.

Re: How to create a list of fields in a table with stats displayed on the Selected Fields sidebar, including rare 10 values, their counts, and percentages?

landen99 — Wed, 09 Sep 2015 15:15:18 GMT

index=myindex | table field1 | appendpipe [| top field1 | eval category="top" ] | appendpipe [| rare field1 | eval category="rare" ] | appendpipe [| stats sum(count) AS count | eval field1="all" | eval percent=100 | eval category="all"] | search category=*

Re: How to create a list of fields in a table with stats displayed on the Selected Fields sidebar, including rare 10 values, their counts, and percentages?

landen99 — Wed, 09 Sep 2015 16:26:14 GMT

This does not give rare values, but it is fast and gives good stats. Still searching for how to add rare values.

| head 99999 | table field1 | fieldsummary | rename count AS count_all | table field count_all values | eval values=replace(values,"\\\\\\\\","\\") | rex field=values max_match=0 "\{(?<values_line>[^,]*,[^,]*)\},?" | fields - values | mvexpand values_line | rex field=values_line max_match=0 "\"value\":\"(?<value>[^\"]*)\",\"count\":(?<count>\d*)" | eval percent=round(count/count_all*100,2) | table field value count percent count_all | head 99 | appendpipe [| stats first(count_all) AS count_all sum(count) AS count first(field) AS field | eval percent=round(count/count_all*100,2) | eval value="top 100 values" | fields - count_all ] | fields - count_all | sort 0 - percent