Security

How can I get the URL path list, visited by the user navigating through a particular application?

rakeshyv0807
Explorer

Hi,

I have a requirement where I need to get the list of URL's the user has navigated through while accessing a particular application. I am able to get the list of URL's but I need to exclude some of these URL's which inlcude .css, .png, .gif, .favicon etc. For example I have the following URL's which were associated for a particular user's session.

https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdf.css
https://example.com/abc/static/6bc3661/release/asdf.gif
https://example.com/abc/static/6bc3661/release/asdfgh.jsp

What I want to achieve is that I need to display only the following URL's in the results and exclude .css, .gif etc.

https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdfgh.jsp

Can some please guide me how I can extract those.

Thanks in advance.

0 Karma
1 Solution

niketn
Legend

@rakeshyv0807, in order to filter unwanted url suffix or retain specific url suffix, you can either create a filter black list or whitelist respectively. It is better to keep this as lookup file in Splunk so that it can be altered/maintained better.

Following is a run anywhere example which lists css,pnj,gif,favicon as balck list and filters out from the search result of URL field using NOT keyword

|  makeresults
|  fields - _time
|  eval URL="https://example.com/abc/index.aspx;https://example.com/abc/region.aspx;https://example.com/abc/static/6bc3661/release/asdf.css;https://example.com/abc/static/6bc3661/release/asdf.gif;https://example.com/abc/static/6bc3661/release/asdfgh.jsp"
|  makemv URL delim=";"
|  mvexpand URL
|  search 
   NOT [| makeresults
    | fields - _time
    | eval filterList="css,pnj,gif,favicon"
    | makemv filterList delim="," 
    | mvexpand filterList
    | eval URL="*.".filterList
    | table URL]

Please try out and confirm. The commands from first | makeresults till | mvexpand URL generates mock data as per the question.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

@rakeshyv0807, in order to filter unwanted url suffix or retain specific url suffix, you can either create a filter black list or whitelist respectively. It is better to keep this as lookup file in Splunk so that it can be altered/maintained better.

Following is a run anywhere example which lists css,pnj,gif,favicon as balck list and filters out from the search result of URL field using NOT keyword

|  makeresults
|  fields - _time
|  eval URL="https://example.com/abc/index.aspx;https://example.com/abc/region.aspx;https://example.com/abc/static/6bc3661/release/asdf.css;https://example.com/abc/static/6bc3661/release/asdf.gif;https://example.com/abc/static/6bc3661/release/asdfgh.jsp"
|  makemv URL delim=";"
|  mvexpand URL
|  search 
   NOT [| makeresults
    | fields - _time
    | eval filterList="css,pnj,gif,favicon"
    | makemv filterList delim="," 
    | mvexpand filterList
    | eval URL="*.".filterList
    | table URL]

Please try out and confirm. The commands from first | makeresults till | mvexpand URL generates mock data as per the question.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

rakeshyv0807
Explorer

Please find below my current query which is extracting URL using eval. Currently I am displaying the list of the URL's with all the suffixes and I want to exclude few suffixes like .css, .gif, .png etc.

sourcetype="audit" subject=abcd  | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime

Can you suggest me how to modify my query to exclude those?

Thanks in advance.

0 Karma

PowerPacked
Builder

can you try this search, for some reason this page is trying to take out the wildcards mentioned in the below search

   sourcetype="audit" subject=abcd  | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime | search URL_Path!=*.css AND URL_Path!=*.gif

Thanks

0 Karma

rakeshyv0807
Explorer

@PowerPacked I have tried adding it and it's still not excluding those.

0 Karma

PowerPacked
Builder

Hi @rakeshyv0807

If the results are extracted into a field called URL --- use the below search

your search | search URL_Path! = *.css AND URL_Path! = *.gif 

Thanks

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...