Subtring from url field and then group using the url

[arunprasadlv]

I have a field "BackendURL" which contains different url's.

for eg :

http://abc.com/emp?name=jim&no=101
http://abc.com/emp?name=tim&no=102
http://gef.com/vehicle

I am trying to generate a report by grouping the url's. Now when i group i want to uniquely identify the backend url , but ignore the queury params (anything after ?). I wrote this rex command to create a new field to substring the value before ? , but it does not work when the url does not contain "?" .

index="idx" Consumer|rex field=BackendURL "^(?.+?)\?"|stats count by BackendURL,url_noparams, host

Thanks and regards
Arun

[somesoni2 · Accepted solution]

Try like this

index="idx" Consumer|rex field=BackendURL  "(?P<requestedUrl>(?P<path>https*:\/\/((?P<contextRoot>[^\/\s]+)\/)?([^\/\s\?;=]+\/)*)((?P<filename>[^\/\s\?;=]+))?|\-)" |stats count by BackendURL,requestedUrl , host

[arunprasadlv]

Thanks a Lot. it worked!!

But I am struggling to understand the logic . Would you pls explain

Regards,
Arun

[somesoni2]

I'm splitting the url to it's different portions

http://abc.com/emp?name=jim&no=101
|--Path------------------|
          |--contextroot-|
                          |-filename-| 

[donB]

@somesoni2 Could you please help if the same url contextPath also has path params?

eg - i can have urls of 2 formats

http://abc.com/emp?name=jim&no=101
http://abc.com/car/ford
http://abc.com/car/tesla

here i need the count of uri paths like below

/car/{id} = 2

/emp = 1  (ignore query params)

[somesoni2]

Give this a try

| makeresults | eval url="http://abc.com/test/emp?name=jim&no=101 http://abc.com/test/car/ford http://abc.com/test/car/tesla" | table url | makemv url | mvexpand url | eval url=replace(url,"\?","/") 
| rename "COMMENT" as "Above code generates sample data. Replace it with your query"
|rex field=url   "(?P<requestedUrl>(?P<path>https*:\/\/((?P<contextRoot>[^\/\s]+)\/)?([^\/\s\?;=]+\/)*)((?P<filename>[^\/\s\?;=]+))?|\-)" | stats count by path

[arunprasadlv]

Got you. Thanks for the quick reply.

[sundareshr]

Try this
| eval baseURL=mvindex(split(BackendURL, "?"), o) | stats count by baseURL