Tip: Sample pfSense Logs Parsed Here

arizvi801 — Tue, 29 Sep 2020 12:39:01 GMT

Hi,

I have parsed some pfSense logs. For anyone making an app, please go ahead and use this info.

Cheers and use in good health.

pfsense_dhcp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+(.*|\[\d+\]))\:\s(?P<dhcp_type>\w+)\s\w+\s(?P<request_address>\d+\.\d+\.\d+\.\d+)\s\w+\s(?P<request_mac>\w+\:\w+\:\w+\:\w+\:\w+\:\w+)\s|\((?P<network>(.*))\)\s\w+\s(?P<interface>\w+)

pfsense_ipv4_icmp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<icmp_type>\w+)\,(?P<icmp_id>\d+)\,(?P<icmp_sequence>\d+)

pfsense_ipv4_tcp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<source_port>\d+)\,(?P<destination_port>\d+)\,(?P<data_length>\d+)\,(?P<tcp_flag>\w+)\,(?P<sequence_number>\d+)\,(?P<ack_number>.*|\d+)\,(?P<tcp_window>\d+)\,\,(?P<tcp_options>.*|\S+)

pfsense_ipv4_udp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<source_port>\d+)\,(?P<destination_port>\d+)\,(?P<data_length>\d+)

pfsense_ipv6_icmpv6

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<class>\d+x\d+)\,(?P<flow_label>\w+)\,(?P<hop_limit>\d+)\,(?P<protocol_text>\w+)\,(?P<protocol_id>\d+)\,(?P<length>\d+)\,(?P<source_address>(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])))\,(?P<destination_address>(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])))\,

pfsense_nginx

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<fqdn>\w+\.\w+\.\w+)\s(?P<logtype>\w+)\:\s(?P<remote_address>\d+\.\d+\.\d+\.\d+)\s\-\s\-\s\[\S+\s\-\d+\]\s\"(?P<request>.*)\"\s(?P<status>\d+)\s(?P<body_bytes_sent>\d+)\s\"(?P<http_referrer>\S+)\"\s\"(?P<http_user_agent>.*)\"

pfsense_snort

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)(.*)\:\s\[(?P<GID>\d+)\:(?P<SID>\d+)\:(?P<rev>\d+)\]\s(?P<description>.*)\s\[Classification:\s(?P<classification>.*)\]\s\[Priority:\s(?P<priority>\d+)\]\s\{(?P<protocol>\w+)\}\s(?P<source_address>\d+\.\d+\.\d+\.\d+)\:(?P<source_port>\d+)\s\-\>\s(?P<destination_address>\d+\.\d+\.\d+\.\d+)\:(?P<destination_port>\d+)

Re: Tip: Sample pfSense Logs Parsed Here

aaraneta_splunk — Wed, 01 Feb 2017 18:24:20 GMT

Hi @arizvi801 - Thank you so much for sharing your pfsense parsing. It would be great if you could put your code in an Answer below that can be accepted. That way other users will know this post is resolved and it can be easily found as a reference 🙂 Thanks!

I've slighted edited your post to make it easier to read all the code. You can click the gear icon to the right of the title and click "Edit" where you can copy and paste all the code into an Answer below.

Re: Tip: Sample pfSense Logs Parsed Here

arizviherjavec — Mon, 23 Apr 2018 17:24:20 GMT

Resolved in Question.

topic Re: Tip: Sample pfSense Logs Parsed Here in Getting Data In

Tip: Sample pfSense Logs Parsed Here

Re: Tip: Sample pfSense Logs Parsed Here

Re: Tip: Sample pfSense Logs Parsed Here