Splunk Search

How do I edit my search to dedup the same web page visited from the same user?

arkonner
Path Finder

I am using the following search to analyze which web pages have been visited.


index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time

Using the dedup command on url_path, the result is a unique page visited over all users - I am looking to dedup the same page visited from the same user


Presently:

10.3.10.10 /
10.3.10.10 /
10.3.10.10 /pageA
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


What I am looking for:

10.3.10.10 /
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


Thank you in advance

Tags (1)
0 Karma

somesoni2
Revered Legend

Only the proper dedup is missing for your search. Try this

 index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time | dedup Client_Address Web_page_Visited

This will the latest entries for each Client_Address Web_page_Visited, if they appear together (based on your sample data, you should get your expected output from this).

0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

somesoni2
Revered Legend

So.. did this search worked?

0 Karma

woodcock
Esteemed Legend

Just add another field; assuming this is your first command:

... | dedup url_path

You just do this:

... | dedup url_path user
0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

woodcock
Esteemed Legend

No problem; did it work for you?

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...