Splunk Search

parse query string parameters

nmarun
Explorer

Our logs will have urls logged in the below manner:

/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&purpose=Billing&pageNumber=1&pageSize=10

These query string params have default values in the API, so they may not all be present in each of the log entries.

https://regex101.com/r/5Ynk4f/1

This is what I've got so far. I need to write in a tabular format:

includeContactsshowOnlyPrimarySitespurposecount
truetruebilling30
falsefalse 50


Thanks

Arun

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

So, this will get your URL parameters into their own fields with their respective values.

| makeresults 
| eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10" 
| rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)" 
| eval params=mvzip(params,values) 
| mvexpand params 
| eval params=split(params,",") 
| eval param=mvindex(params,0), {param}=mvindex(params,1)
| fields - param values params 
| stats values(*) as * by url

After that, what you will end up with is a stats command that groups by an unknown set of fields.  That is not possible.  The by clause of stats must be a list of field names, not a wildcard.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust
What is your question? You appear to have the URL parsed already. If you need additional parsing, check out the URLToolbox app on splunkbase.
---
If this reply helps you, Karma would be appreciated.
0 Karma

nmarun
Explorer

@richgalloway,

How to render it into a table after parsing?

eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10"
|rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)"| stats count by params

The highlighted part is what I'm trying to figure out.

Thanks

Arun

0 Karma

richgalloway
SplunkTrust
SplunkTrust

So, this will get your URL parameters into their own fields with their respective values.

| makeresults 
| eval url="url=/v1/customers/1/sites?includeContacts=True&showOnlyPrimarySites=True&pageNumber=1&pageSize=10" 
| rex field=url max_match=0 "[\?\&](?<params>[^=]+)=(?<values>[^&]+)" 
| eval params=mvzip(params,values) 
| mvexpand params 
| eval params=split(params,",") 
| eval param=mvindex(params,0), {param}=mvindex(params,1)
| fields - param values params 
| stats values(*) as * by url

After that, what you will end up with is a stats command that groups by an unknown set of fields.  That is not possible.  The by clause of stats must be a list of field names, not a wildcard.

---
If this reply helps you, Karma would be appreciated.
0 Karma

nmarun
Explorer

@richgalloway,

Yes, that's my question - is there a way to split the params and values array so I run stats on them?

Thanks,

Aru

0 Karma

nmarun
Explorer

@richgalloway, Thanks so much sir.

Arun

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...