Tip: Sample pfSense Logs Parsed Here

[arizvi801]

Hi,

I have parsed some pfSense logs. For anyone making an app, please go ahead and use this info.

Cheers and use in good health.

pfsense_dhcp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+(.*|\[\d+\]))\:\s(?P<dhcp_type>\w+)\s\w+\s(?P<request_address>\d+\.\d+\.\d+\.\d+)\s\w+\s(?P<request_mac>\w+\:\w+\:\w+\:\w+\:\w+\:\w+)\s|\((?P<network>(.*))\)\s\w+\s(?P<interface>\w+)

pfsense_ipv4_icmp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<icmp_type>\w+)\,(?P<icmp_id>\d+)\,(?P<icmp_sequence>\d+)

pfsense_ipv4_tcp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<source_port>\d+)\,(?P<destination_port>\d+)\,(?P<data_length>\d+)\,(?P<tcp_flag>\w+)\,(?P<sequence_number>\d+)\,(?P<ack_number>.*|\d+)\,(?P<tcp_window>\d+)\,\,(?P<tcp_options>.*|\S+)

pfsense_ipv4_udp

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<tos>\d+x\d+)\,\,(?P<ttl>\d+)\,(?P<id>\d+)\,(?P<offset>\d+)\,(?P<flags>\w+)\,(?P<protocol_id>\d+)\,(?P<protocol_text>\w+)\,(?P<length>\d+)\,(?P<source_address>\d+\.\d+\.\d+\.\d+)\,(?P<destination_address>\d+\.\d+\.\d+\.\d+)\,(?P<source_port>\d+)\,(?P<destination_port>\d+)\,(?P<data_length>\d+)

pfsense_ipv6_icmpv6

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+.*|\s\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)\:\s(?P<rule_number>.*|\d+)\,(?P<sub_rule_number>.*|\d+)\,(?P<anchor>.*|\w+)\,(?P<tracker>\d+)\,(?P<real_interface>\w+)\,(?P<reason>\w+)\,(?P<action>\w+)\,(?P<direction>\w+)\,(?P<ip_version>\d)\,(?P<class>\d+x\d+)\,(?P<flow_label>\w+)\,(?P<hop_limit>\d+)\,(?P<protocol_text>\w+)\,(?P<protocol_id>\d+)\,(?P<length>\d+)\,(?P<source_address>(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])))\,(?P<destination_address>(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])))\,

pfsense_nginx

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<fqdn>\w+\.\w+\.\w+)\s(?P<logtype>\w+)\:\s(?P<remote_address>\d+\.\d+\.\d+\.\d+)\s\-\s\-\s\[\S+\s\-\d+\]\s\"(?P<request>.*)\"\s(?P<status>\d+)\s(?P<body_bytes_sent>\d+)\s\"(?P<http_referrer>\S+)\"\s\"(?P<http_user_agent>.*)\"

pfsense_snort

(?P<host>\d+\.\d+\.\d+\.\d+)\s(?P<timestamp>\S+(.*|\s)\s\d+\s\d+\:\d+\:\d+)\s(?P<logtype>\w+)(.*)\:\s\[(?P<GID>\d+)\:(?P<SID>\d+)\:(?P<rev>\d+)\]\s(?P<description>.*)\s\[Classification:\s(?P<classification>.*)\]\s\[Priority:\s(?P<priority>\d+)\]\s\{(?P<protocol>\w+)\}\s(?P<source_address>\d+\.\d+\.\d+\.\d+)\:(?P<source_port>\d+)\s\-\>\s(?P<destination_address>\d+\.\d+\.\d+\.\d+)\:(?P<destination_port>\d+)

[arizviherjavec · Accepted solution]

Resolved in Question.

[aaraneta_splunk]

Hi @arizvi801 - Thank you so much for sharing your pfsense parsing. It would be great if you could put your code in an Answer below that can be accepted. That way other users will know this post is resolved and it can be easily found as a reference 🙂 Thanks!

I've slighted edited your post to make it easier to read all the code. You can click the gear icon to the right of the title and click "Edit" where you can copy and paste all the code into an Answer below.