Multiple URL's response time by URL's

[shabdadev]

Hi All,

I have this scenario where i have couple of diff types of URL's like

/webapp/wcs/services/Key
/webapp/wcs/services/Address
/wcs/resources/v1?*
/webapp/orderId=*

I want to write a query to list down the response time for each url by url wise like this

URL                                                                                    Response time
/webapp/wcs/services/Key                                              50
/webapp/wcs/services/Address                                      30
/wcs/resources/v1?*                                                         25
/webapp/orderId=*                                                           10

I know we can use like command but like only matches the exact url which are starightforward , if wildcard is there its not able to match and hence not showing the stats , how to manipulate the query to get the required result .....i have lot of url's but only listing 4 here .
i wrote the below query but its not returning the data for the url's containing the wildcards

index=ih host=los  sourcetype=ihD 
(URLRedefined="/webapp/wcs/services/Key"   OR
URLRedefined="/webapp/wcs/services/Address"   OR
URLRedefined="/wcs/resources/v1?*"  OR URLRedefined="/webapp/orderId=*"  )
| eval URL=case(like(URLRedefined,"/webapp/wcs/services/Key"),"/webapp/wcs/services/Key",
                like(URLRedefined,"/webapp/wcs/services/Address"),"/webapp/wcs/services/Address",
                like(URLRedefined,"/wcs/resources/v1?*"),"/wcs/resources/v1?*",
                like(URLRedefined,"/webapp/orderId=*"),"/webapp/orderId=*")
|  stats perc95(ResponseTime)  by URL

[niketn · Accepted solution]

I have changed from like to match (like will return same results). You should escape question mark (?) in the match pattern with forward slash. Following is run anywhere search to test your eval statement.

| makeresults 
| eval URLRedefined="/wcs/resources/v1\?*"
| eval URL=case(like(URLRedefined,"/webapp/wcs/services/Key"),"/webapp/wcs/services/Key",
                 like(URLRedefined,"/webapp/wcs/services/Address"),"/webapp/wcs/services/Address",
                 like(URLRedefined,"/wcs/resources/v1\?*"),"/wcs/resources/v1?*",
                 like(URLRedefined,"/webapp/orderId=*"),"/webapp/orderId=*")

Or

| makeresults 
| eval URLRedefined="/wcs/resources/v1\?*"
| eval URL=case(match(URLRedefined,"/webapp/wcs/services/Key"),"/webapp/wcs/services/Key",
                 match(URLRedefined,"/webapp/wcs/services/Address"),"/webapp/wcs/services/Address",
                 match(URLRedefined,"/wcs/resources/v1\?*"),"/wcs/resources/v1?*",
                 match(URLRedefined,"/webapp/orderId=*"),"/webapp/orderId=*")

Please try out and let us know.

[shabdadev]

Awesome it worked Niket .
One more question :

1. In match its optional or what to give the false condition ??
2. Why forward slash ...why splunk cant accept ? mark in the query ??

[niketn]

1) With case you can always add either 1==1 or true() condition in the end to make sure it handles the default scenario where none of defined tests match.

For example:

 | eval URL=case(match(URLRedefined,"/webapp/wcs/services/Key"),"/webapp/wcs/services/Key",
                  match(URLRedefined,"/webapp/wcs/services/Address"),"/webapp/wcs/services/Address",
                  match(URLRedefined,"/wcs/resources/v1\?*"),"/wcs/resources/v1?*",
                  match(URLRedefined,"/webapp/orderId=*"),"/webapp/orderId=*",
                  true(),"unKnownURL")

2) Depending upon the type of SPL functionality being used and values being passed some characters have different implications hence they need to be escaped (there are different ways of escaping as well). Most common way while dealing with regular expressions, is to add forward slash before a special character. This treats the character as is and override its other meaning. Having said this, in current scenario both like() and match() are pattern based where question mark (?) implies field name being extracted. Hence it needs to be escaped. Refer to Regular Expression details on regex101.com

[shabdadev]

Thanks Niket ,
I have one more query
i am running this below query :

index=ihs host=losat  sourcetype=ih 
( URLRedefined="/wcs/resources/products/*/credit/fin/v1\?brand=*" OR URLRedefined="/webapp/wcs/OrderItem\?orderId=*")

| eval URL=case(match(URLRedefined,"/wcs/resources/products/*/credit/fin/v1\?brand=*"),"/wcs/resources/products/*/credit/fin/v1?brand=*",
                match(URLRedefined,"/webapp/wcs/OrderItem\?orderId=*"),"/webapp/wcs/OrderItem?orderId=*"  )
| stats count perc95(ResponseTime)  by URL 

I am getting stats for the orderitem url but for the credit/fin url i am not getting stats ....i am not getting what is wrong here .

[niketn]

@shabdadev, you would need to match only credit URL to test whether the pattern is correct as per the actual URLs. Since we have filtered only two types or URLs even if orderitem URL is working fine for you, you can treat everything else as credit URL.

However, this is the correct pattern for credit URL

 | eval URL=case(match(URLRedefined,"\/wcs\/resources\/products\/.*\/credit\/fin\/v1\?brand=*"),"/wcs/resources/products/*/credit/fin/v1?brand=*",
                 match(URLRedefined,"/webapp/wcs/OrderItem\?orderId=*"),"/webapp/wcs/OrderItem?orderId=*"  )