Splunk Search

parse query string parameters

nmarun
Explorer

Our logs will have urls logged in the below manner:

/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&purpose=Billing&pageNumber=1&pageSize=10

These query string params have default values in the API, so they may not all be present in each of the log entries.

https://regex101.com/r/5Ynk4f/1

This is what I've got so far. I need to write in a tabular format:

includeContactsshowOnlyPrimarySitespurposecount
truetruebilling30
falsefalse 50


Thanks

Arun

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

So, this will get your URL parameters into their own fields with their respective values.

| makeresults 
| eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10" 
| rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)" 
| eval params=mvzip(params,values) 
| mvexpand params 
| eval params=split(params,",") 
| eval param=mvindex(params,0), {param}=mvindex(params,1)
| fields - param values params 
| stats values(*) as * by url

After that, what you will end up with is a stats command that groups by an unknown set of fields.  That is not possible.  The by clause of stats must be a list of field names, not a wildcard.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust
What is your question? You appear to have the URL parsed already. If you need additional parsing, check out the URLToolbox app on splunkbase.
---
If this reply helps you, Karma would be appreciated.
0 Karma

nmarun
Explorer

@richgalloway,

How to render it into a table after parsing?

eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10"
|rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)"| stats count by params

The highlighted part is what I'm trying to figure out.

Thanks

Arun

0 Karma

richgalloway
SplunkTrust
SplunkTrust

So, this will get your URL parameters into their own fields with their respective values.

| makeresults 
| eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10" 
| rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)" 
| eval params=mvzip(params,values) 
| mvexpand params 
| eval params=split(params,",") 
| eval param=mvindex(params,0), {param}=mvindex(params,1)
| fields - param values params 
| stats values(*) as * by url

After that, what you will end up with is a stats command that groups by an unknown set of fields.  That is not possible.  The by clause of stats must be a list of field names, not a wildcard.

---
If this reply helps you, Karma would be appreciated.
0 Karma

nmarun
Explorer

@richgalloway,

Yes, that's my question - is there a way to split the params and values array so I run stats on them?

Thanks,

Aru

0 Karma

nmarun
Explorer

@richgalloway, Thanks so much sir.

Arun

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...