Getting Data In

web gateway filter activity to urls

daveevad
New Member

i have 117 sites listed from homeland security. i need to check if any of our machine have visited them. We have McAfee web gateway logs funneled into splunk. What's the best way to go about looking for that activity?

Labels (1)
Tags (1)
0 Karma
1 Solution

manjunathmeti
Champion

You can create a csv file with all the 117 urls, upload it in search head and use that in your search. This will filter data with all the urls in csv file.

urls.csv

dhost
www.upanddown.ocry.com
www.upanddown.ocry.com
good.weascapes.com
khinhte.chinhsech.com
...

Search query:

source=Webgateway [ | inputlookup urls.csv]

View solution in original post

0 Karma

PavelP
Motivator

such lists (example: https://urlhaus.abuse.ch/browse/) often contains full urls with http:// prefix. A url http://www.example.com/ from the csv file will not match https://www.example.com/foo?xx in your proxy log. It is better to extract the domain part (www.example.com or even example.com) before searching.

0 Karma

manjunathmeti
Champion

You can create a csv file with all the 117 urls, upload it in search head and use that in your search. This will filter data with all the urls in csv file.

urls.csv

dhost
www.upanddown.ocry.com
www.upanddown.ocry.com
good.weascapes.com
khinhte.chinhsech.com
...

Search query:

source=Webgateway [ | inputlookup urls.csv]
0 Karma

daveevad
New Member

cool! thanks!

0 Karma

daveevad
New Member

ok, maybe i was making this harder than it needs to be. I can do this for example...
source=Webgateway walmart.com
i get alot of hits, of course. Do i have to do this 117 times, one for each url/ftp site?

0 Karma

daveevad
New Member

can i use "OR" statements between urls to search for several at one time?

0 Karma

daveevad
New Member

Ah, answer is yes. How many can i string together?

0 Karma

daveevad
New Member

apparently at least 10 sites. Was able to do searching without errors. Talked myself through this. I good now. Open to better ways though!
Thanks.

0 Karma

manjunathmeti
Champion

Are these website values part of a field in the data?

0 Karma

daveevad
New Member

as in my example for a walmart.com search, it showed up as
dhost="beacon.walmart.com"
That what you mean?

0 Karma

manjunathmeti
Champion

yes, are other website names are part of this field?

Provide some sample for below query.

source=Webgateway | head 20 | table dhost
0 Karma

daveevad
New Member

i had 117 various urls and ftp sites. I was checking to see if any of our user went to any of those site. One of my queries looked like this.
source=Webgateway www.upanddown.ocry.com OR dothi.chinhsech.com OR good.weascapes.com OR khinhte.chinhsech.com OR hcm.vozforumsx.com OR image.biengioivn.com OR lat.conglyan.com OR login.chinhphuna.com OR login.haiduongpcg.com OR luan.conglyan.com
Which worked, i threw in a youtube.com and a walmart.com in there to check. A table would be cleaner though...

0 Karma

manjunathmeti
Champion

I converted my comment to answer. Please accepts it if it works for you.

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...