Splunk Search

How do I edit my search to dedup the same web page visited from the same user?

arkonner
Path Finder

I am using the following search to analyze which web pages have been visited.


index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time

Using the dedup command on url_path, the result is a unique page visited over all users - I am looking to dedup the same page visited from the same user


Presently:

10.3.10.10 /
10.3.10.10 /
10.3.10.10 /pageA
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


What I am looking for:

10.3.10.10 /
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


Thank you in advance

Tags (1)
0 Karma

somesoni2
Revered Legend

Only the proper dedup is missing for your search. Try this

 index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time | dedup Client_Address Web_page_Visited

This will the latest entries for each Client_Address Web_page_Visited, if they appear together (based on your sample data, you should get your expected output from this).

0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

somesoni2
Revered Legend

So.. did this search worked?

0 Karma

woodcock
Esteemed Legend

Just add another field; assuming this is your first command:

... | dedup url_path

You just do this:

... | dedup url_path user
0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

woodcock
Esteemed Legend

No problem; did it work for you?

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...