Splunk Search

count by multiple instances of same field name in one request

ssorlie
New Member

I'm using Splunk to interrogate web logs. Users of our site can select one or more
parameters in their data requests, identified as p=. I want to get a count by
parameter; however, when I use stats count by p, Splunk only picks up the first p=<> in the
list. The others are ignored. Therefore to get the information, I'm performing searches by pattern (essentially a regex) but it's time consuming since there are 200+ parameters. Is there another way? Or do I need to write a script?

p=grid_id&p=swv_dwn&p=ret_tlt0&p=no_sun1&p=T10M&p=wspd10arpt&p=RAIN&p=srf_alb

Tags (2)
0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

Understanding that you only get the value for the first "p" field, then it will be important to understand that Splunk will match key value pairs using the "=" character by default. What Splunk does not do by default is to allow multiple field values for the same meta field in the same event.

The easiest way is to extract all values for field "p" and then allow for multi-values for each event. Assume that your data lands according to the sample above:

alt text

Use an entry in props.conf to indicate the requirement for a REPORT. Why are REPORT? It is because we are looking for fields created at search time. In this case a simple entry like so will do:

[answers-1371406335]
REPORT-get_p_fields = get_p_fields

The props.conf entry directs Splunk to search for a matching stanza in transforms.conf. That is where you establish the required regular expression, match it to a field name and indicate that there are multiple values for the same entry allowed for each event -like this:

[get_p_fields]
REGEX  = p\=(.+?)\&
FORMAT = p::$1
MV_ADD = true

This will create a list of items for each event.

alt text

And, finally, you enumerate with stats by counting the field "p".

alt text

BTW: If you want to manipulate the items in the list, you will expand the list per the individual values using mvexpand.

alt text

TIP: If you do not know where to place the props.conf or transforms.conf, then use the Search App. You will find this under $SPLUNK_HOME/etc/apps/search/local or %SPLUNK_HOME%\etc\apps\search\local.

lguinn2
Legend

What you have listed is not valid Splunk search syntax, so perhaps that is the problem. Your search should be:

p=grid_id OR p=swv_dwn OR p=ret_tlt0 OR p=no_sun1 OR p=T10M OR p=wspd10arpt OR p=RAIN OR p=srf_alb
| stats count by p

where the items in the first line of the search are the selected values for p.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...