topic Need help with regex, mvindex, or other option for field-extractions in Splunk Search

Need help with regex, mvindex, or other option for field-extractions

avishek_08 — Tue, 29 Sep 2020 17:04:44 GMT

Dec 5 18:04:51 192.168.69.50 pfsp: Host Detection alert #22049413, start 2017-12-06 00:03:45 GMT, duration 66, direction incoming, host 71.92.104.13, signatures (ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification), impact 4.00 Gbps/386.20 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")
Dec 5 16:19:51 192.168.69.50 pfsp: Host Detection alert #22049331, start 2017-12-05 22:16:45 GMT, duration 186, direction incoming, host 24.177.66.30, signatures (ICMP, IP Fragmentation, TCP NULL, TCP SYN, TCP RST, Total Traffic, UDP), impact 310.64 Mbps/104.45 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")

Please help me parse this log event. For some reason, it has been parsed as a single field in my Splunk instance. On top of it all, I do not have access to .prof files to change the field extraction criteria hence need to utilize Splunk commands to properly parse them to create a table.
Please help me parse the log as following>> _time: Dec 5 18:04:51, collector IP_address: 192.168.69.50, Type: Host Detection, Alert_id: 22049413, start_time: 2017-12-06 00:03:45 GMT, duration: 66, direction: incoming, host: 71.92.104.13, signatures: ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification, impact: 4.00 Gbps/386.20 Kpps, importance: 2, managed_objects: Tonga, parent managed object: nil.

I have been trying to use Rex commands but since I am not an expert at it. Its not working out so much. Colleague suggested mvindex, I am trying to parse it using that but I hope I can get some help from awesome Splunk community.

Re: Need help with regex, mvindex, or other option for field-extractions

harsmarvania57 — Wed, 06 Dec 2017 04:33:44 GMT

Hi @avishek_08,

Will you please try this regex ?

<your search> | rex "(?m)^(?<time>.*?)\s(?<collector_IP_Address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?<Type>.*)#(?<Alert_id>.*),\sstart\s(?<start_time>.*),\sduration\s(?<duration>\d+),\sdirection\s(?<direction>\w+),\shost\s(?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?<signatures>.*)\),\simpact\s(?<impact>.*),\simportance\s(?<importance>\d+),\smanaged_objects\s\(\"(?<managed_objects>\w+)\"\),\s\(parent\smanaged\sobject\s\"(?<parent_managed_object>\w+)\"\)"

For reference please check regex with sample data at https://regex101.com/r/IOZJxI/2/

I hope this helps.

Thanks,
Harshil

Re: Need help with regex, mvindex, or other option for field-extractions

avishek_08 — Wed, 06 Dec 2017 17:13:14 GMT

Thank you so much Harshil. Let me show you the query I am entering and maybe you can help me pinpoint my error.

index=tonga_logs host=192.168.69.50 "Host Detection" 
| rex "(?m)^(?.*?)\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?.*)#(?.*),\sstart\s(?.*),\sduration\s(?\d+),\sdirection\s(?\w+),\shost\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?.*)\),\simpact\s(?.*),\simportance\s(?\d+),\smanaged_objects\s\(\"(?\w+)\"\),\s\(parent\smanaged\sobject\s\"(?\w+)\"\)"
|table time, collector_IP_Address, Type, Alert_id, start_time, duration, direction, host, signatures, impact, importance, managed_objects

I feel like everything is there but it still doesn't display anything for me.

Re: Need help with regex, mvindex, or other option for field-extractions

harsmarvania57 — Wed, 06 Dec 2017 17:55:41 GMT

Can you please post your query and sample data in Code Sample format (Use Button 101010)

Re: Need help with regex, mvindex, or other option for field-extractions

avishek_08 — Wed, 06 Dec 2017 20:42:25 GMT

If you don't mind me asking: could you direct me how to do that?

Re: Need help with regex, mvindex, or other option for field-extractions

niketn — Wed, 06 Dec 2017 21:17:15 GMT

When you start typing in the Text Box here on Splunk Answers you will see an icon 101010 which is the Code button. This will prevent special characters in your code from being escaped.

Alternatively, when you start typing in your SPL, you can prefix four spaces just before each new line of your code.

Re: Need help with regex, mvindex, or other option for field-extractions

ppablo — Fri, 12 Jan 2018 20:43:08 GMT

Thanks for sharing that tip with them @niketnilay 🙂 I've just gone ahead and reformatted it in a Code Sample box already so all special characters are visible.