topic Re: In a search, how do you group similar URL patterns? in Splunk Search

In a search, how do you group similar URL patterns?

alex129 — Wed, 03 Oct 2018 14:29:43 GMT

I am doing a search and evaluating count, avg RT based on some URL patterns. Below are the URLs for my category pages :

//Food/Back-to-School/Breakfast/Breakfast-Bars/c/1015001004
//Food/Back-to-School/Breakfast/Cereal/c/1015001001
//Home-%26-Lifestyle/Beauty/Cosmetics/Eyes/c/2004001001

Using below query gives me the different results for different URLs, but I want results grouped into 1 Relative URL.

sourcetype=apache:access request_uri="\*/c/\*"  |  search apache_responsetime=* | chart perc90(apache_responsetime) over request_uri

What query I should use to get desired results?

Re: In a search, how do you group similar URL patterns?

VatsalJagani — Wed, 03 Oct 2018 18:30:35 GMT

@alex129, Can you explain with example what do you mean by similar url?

Re: In a search, how do you group similar URL patterns?

alex129 — Wed, 03 Oct 2018 18:48:24 GMT

@VatsalJagani , the URLs mentioned in my questions are for category page & can be searched as requesturi="/c/"

Re: In a search, how do you group similar URL patterns?

burwell — Wed, 03 Oct 2018 19:34:16 GMT

So if they have /c/ they are the same? What are other url patterns?

Re: In a search, how do you group similar URL patterns?

somesoni2 — Wed, 03 Oct 2018 19:44:07 GMT

Since you're looking at only specific type of URIs (only one pattern), you can simply do something like this

sourcetype=apache:access request_uri="\*/c/\*"  apache_responsetime=* 
| stats perc90(apache_responsetime)  | eval request_uri="*/c/*" | table requiest_uri perc90(apache_responsetime)

A more generic approach would be this (where you can look for multiple URI patterns)

sourcetype=apache:access (request_uri="\*/a/\*" OR   request_uri="\*/b/\*" OR...other patterns ) apache_responsetime=* 
| eval request_uri=case(searchmatch(request_uri="\*/a/\*"),"*/a/*", request_uri="\*/b/\*"),"*/b/*",..literal/static values for other patterns..)
| chart perc90(apache_responsetime) over request_uri

Re: In a search, how do you group similar URL patterns?

burwell — Wed, 03 Oct 2018 19:51:15 GMT

So I recommend creating a new field which normalizes the request uris. Also you don't need a separate search apache_responsetime=*. Just combine it. The default is to AND the searches.

It is not clear what the space of your URLs is. If you want to lump all Food and all Home life style pages together then maybe something like:

 sourcetype=apache:access request_uri="\*/c/\*"  apache_responsetime=* | rex field=request_uri "//(?<category>\S+)/" | chart perc90(apache_responsetime) by category

Re: In a search, how do you group similar URL patterns?

alex129 — Wed, 03 Oct 2018 19:52:57 GMT

@burwell
I mean "/c/" is common in all category URLs and I need to group all these URLs so that I can get count, response time etc. for grouped URL

Re: In a search, how do you group similar URL patterns?

alex129 — Wed, 03 Oct 2018 20:18:17 GMT

@somesoni2 ,

Your 1st query, I am getting the results for 90th percentile in 1 column, But
1. Not getting chart over time for grouped URL
2. how to get stats count for all these URLs

Re: In a search, how do you group similar URL patterns?

alex129 — Wed, 03 Oct 2018 20:20:06 GMT

@burwell

Your query is grouping all URLs post '/c/' i.e.

/Food/Back-to-School/Breakfast/Breakfast-Bars/c/1015001001
/Food/Back-to-School/Breakfast/Breakfast-Bars/c/1015001002
/Food/Back-to-School/Breakfast/Breakfast-Bars/c/1015001003

into 

/Food/Back-to-School/Breakfast/Breakfast-Bars/c/

I need all URLs before /c/ as well to group into 1 URL