topic Re: How to capture URL information in Splunk Search

How to capture URL information

bagarwal — Tue, 25 Oct 2016 11:15:34 GMT

Hello All,

I want to create a report for top 10 URL's visited by the users. However, when I see the events in PaloAlto Firewall , I don't see any fields containing URL information though there is URL category field.

e.g. in URL category field I am getting as "computer -and internet-info" , but I want specific URL information e.g. *.dell.com or *.net or *.saas.hp.com/ something like this.

Can any please help how to get the URL information in firewall events so I can pull the data and create the report.

Thanks in advance

Binay Agarwal

Re: How to capture URL information

skoelpin — Tue, 25 Oct 2016 11:50:26 GMT

Hello @bagarwal, you will need to extract the field using a regular expression. Post a sample of your data and I will help you write the search

Re: How to capture URL information

bagarwal — Tue, 25 Oct 2016 15:04:41 GMT

Hello Skoelpin,

Thank You for your response.

Here is the 2 sample data: Just have replaced some information with <>.

Hope it helps to extract the URL field using a regular expression . If not, please let me know any specific sample you need.

========================================================

Thanks & Regards,
Binay Agarwal

Re: How to capture URL information

btorresgil — Tue, 29 Sep 2020 11:35:58 GMT

Hello,

To get URL's in Splunk from a Palo Alto Networks Next-generation Firewall, you need to send URL logs to Splunk:

Install a URL-Filtering license on the firewall
Create a URL-Filtering security profile with all categories set to 'alert' or some other action besides 'allow' (allow does not produce a log)
Assign the URL-Filtering profile to a security rule that sees the traffic you want to log.
Assign the Log Forwarding profile you created for Splunk to the same rule.
Commit the configuration
Assuming you installed the Palo Alto Networks Add-on for Splunk, view the URL logs with this search:

eventtype=pan log_subtype=url | table dest_hostname url

Re: How to capture URL information

bagarwal — Fri, 28 Oct 2016 09:59:23 GMT

Hello @skoelpin ,

Can you please help in writing the regex or do you need any more details.

Thanks & Regards,
Binay Agarwal

Re: How to capture URL information

bagarwal — Fri, 28 Oct 2016 10:01:00 GMT

Hi @btorresgil,

Thank You for your response. Will try this also 🙂 However, would be prefer to get the URL links and view without using Palo Alto Networks App.

Thanks & Regards,
Binay Agarwal

Re: How to capture URL information

btorresgil — Fri, 28 Oct 2016 17:19:33 GMT

Hi Binay, you don't need to use the App, just the Add-on. The Add-on simply contains an optimized props.conf and transforms.conf for parsing the default Palo Alto Networks logs. It will not slow down your Splunk instance, it just does all the parsing work for you so you don't have to create a parser or a custom log format. Creating a regex yourself would by much slower to process every log than the methods used in the Add-on.

Palo Alto Networks Add-on:
https://splunkbase.splunk.com/app/2757/

Re: How to capture URL information

pj — Wed, 29 Nov 2017 21:58:49 GMT

To add -

In order to forward URL logs, it is necessary to forward Threat logs of Severity 'informational' to the Syslog server on the PaloAlto server.