topic Re: How can I parse Snort logs from pfsense syslog? in Getting Data In

How can I parse Snort logs from pfsense syslog?

TribanMD — Fri, 18 Jan 2013 03:09:50 GMT

I was able to set Splunk up to configure the reports for the pfsense firewall logs. But I would also like to create a similar report for just the snort logs. Right now they are being set into the pfsense system log. I can view them by just using the keyword "snort" in the search on the specific source, but I would like to parse out the fields as well. I would like to then try throwing that data into the Google Maps App. Any ideas?

Ok, so the logs are showing up in Splunk from pfsense in the following format:

(snort log alet)

Jan 19 10:53:25 SplunkSourceHost Jan 19 10:53:24 snort[61858]: [120:6:1] (http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED [Classification: Unknown Traffic] [Priority: 3] {TCP} SRC_IP:PRT -> DST_IP:PORT

Santized, so after the SplunkSourceHost is the log from pfsense. In this case it is the log from the snort service in pfsense. Firewall logs look like this:

(pfSense firewall block)

Jan 19 15:34:29 SplunkSourceHost Jan 19 15:34:28 pf: 00:00:10.461152 rule 1/0(match): block in on em0: (tos 0x20, ttl 95, id 256, offset 0, flags [none], proto TCP (6), length 40)
Jan 19 15:34:29 SplunkSourceHost Jan 19 15:34:28 pf:     SRC_IP.PORT > DST_IP.PORT: Flags [S], cksum 0x4302 (correct), seq 1609564160, win 16384, length 0

I used the guide here http://www.seattleit.net/blog/tag/pfsense/ to configure the transforms and props files. I imagine I would need to do something similar to format the snort logs. Just not sure how.

Thanks for any help you can provide.

Re: How can I parse Snort logs from pfsense syslog?

Ayn — Fri, 18 Jan 2013 07:50:00 GMT

Log samples please?

Re: How can I parse Snort logs from pfsense syslog?

TribanMD — Mon, 21 Jan 2013 00:29:27 GMT

posted, let me know if you are looking for something different. Thanks!

Re: How can I parse Snort logs from pfsense syslog?

sbrant_splunk — Sun, 05 May 2013 20:04:25 GMT

Add the following to your configuration files for pfsense:

------- transforms.conf

###### snort ######

[force_sourcetype_for_snort]
DEST_KEY = MetaData:Sourcetype
REGEX = \w+\s+\d+\s+\d+\:\d+\:\d+\s+[^\s]+\s+snort\[\d+\]\:
FORMAT = sourcetype::snort

[category_for_snort]
REGEX = Classification\:\s+([^\]]+)
FORMAT = category::"$1"

[dest_ip_for_snort]
REGEX = \-\>\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
FORMAT = dest_ip::$1

[dest_port_for_snort]
REGEX = \-\>\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\:(\d+)
FORMAT = dest_port::$1

[pid_for_snort]
REGEX = snort\[(\d+)
FORMAT = pid::$1

[severity_id_for_snort]
REGEX = Priority\:\s+(\d+)
FORMAT = severity_id::$1

[signature_for_snort]
REGEX = snort\[\d+\]\:\s+\[[^\]]+\]\s+(.*?)(\s+\[Classification|\[Priority)
FORMAT = signature::"$1"

[signature_id_for_snort]
REGEX = snort\[\d+\]\:\s+\[([^\]]+)
FORMAT = signature_id::"$1"

[src_ip_for_snort]
REGEX = \{\w+\}\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
FORMAT = src_ip::$1

[src_port_for_snort]
REGEX = \{\w+\}\s+\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\:(\d+)
FORMAT = src_port::$1

[transport_for_snort]
REGEX = \{([^\}]+)
FORMAT = transport::$1

------- props.conf

[source::udp:514]  # --- May need to change this source, depending on how you're collecting the data
TRANSFORMS-force_sourcetype_for_snort = force_sourcetype_for_snort

[snort]
SHOULD_LINEMERGE=false
REPORT-category_for_snort = category_for_snort
REPORT-dest_ip_for_snort = dest_ip_for_snort
REPORT-dest_port_for_snort = dest_port_for_snort
REPORT-pid_for_snort = pid_for_snort
REPORT-0severity_id_for_snort = severity_id_for_snort
REPORT-signature_for_snort = signature_for_snort
REPORT-signature_id_for_snort = signature_id_for_snort
REPORT-src_ip_for_snort = src_ip_for_snort
REPORT-src_port_for_snort = src_port_for_snort
REPORT-transport_for_snort = transport_for_snort

Re: How can I parse Snort logs from pfsense syslog?

TribanMD — Tue, 07 May 2013 02:06:44 GMT

Are the confs reversed? My current props has the reports/transforms data while my Transforms has the regexs and such (for the pfsense-firewall sources). Also another problem is that I can't seem to send pfsense snort data separately, all or nothing. so all logs come over syslog from pfsense. Otherwise I can use the Snort for Splunk app.

Re: How can I parse Snort logs from pfsense syslog?

sbrant_splunk — Tue, 07 May 2013 02:21:34 GMT

Yep, you're right. I just changed them around.

Re: How can I parse Snort logs from pfsense syslog?

sbrant_splunk — Tue, 07 May 2013 02:36:58 GMT

Do you have Splunk listening on port 514? If so, the first stanza in props.conf should force the sourcetype of snort on just the snort logs from the input.