Splunk Search

Need help with regex, mvindex, or other option for field-extractions

New Member
  1. Dec 5 18:04:51 192.168.69.50 pfsp: Host Detection alert #22049413, start 2017-12-06 00:03:45 GMT, duration 66, direction incoming, host 71.92.104.13, signatures (ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification), impact 4.00 Gbps/386.20 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")
  2. Dec 5 16:19:51 192.168.69.50 pfsp: Host Detection alert #22049331, start 2017-12-05 22:16:45 GMT, duration 186, direction incoming, host 24.177.66.30, signatures (ICMP, IP Fragmentation, TCP NULL, TCP SYN, TCP RST, Total Traffic, UDP), impact 310.64 Mbps/104.45 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")

Please help me parse this log event. For some reason, it has been parsed as a single field in my Splunk instance. On top of it all, I do not have access to .prof files to change the field extraction criteria hence need to utilize Splunk commands to properly parse them to create a table.
Please help me parse the log as following>> _time: Dec 5 18:04:51, collector IP_address: 192.168.69.50, Type: Host Detection, Alert_id: 22049413, start_time: 2017-12-06 00:03:45 GMT, duration: 66, direction: incoming, host: 71.92.104.13, signatures: ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification, impact: 4.00 Gbps/386.20 Kpps, importance: 2, managed_objects: Tonga, parent managed object: nil.

I have been trying to use Rex commands but since I am not an expert at it. Its not working out so much. Colleague suggested mvindex, I am trying to parse it using that but I hope I can get some help from awesome Splunk community.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi @avishek_08,

Will you please try this regex ?

<your search> | rex "(?m)^(?<time>.*?)\s(?<collector_IP_Address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?<Type>.*)#(?<Alert_id>.*),\sstart\s(?<start_time>.*),\sduration\s(?<duration>\d+),\sdirection\s(?<direction>\w+),\shost\s(?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?<signatures>.*)\),\simpact\s(?<impact>.*),\simportance\s(?<importance>\d+),\smanaged_objects\s\(\"(?<managed_objects>\w+)\"\),\s\(parent\smanaged\sobject\s\"(?<parent_managed_object>\w+)\"\)"

For reference please check regex with sample data at https://regex101.com/r/IOZJxI/2/

I hope this helps.

Thanks,
Harshil

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Hi @avishek_08,

Will you please try this regex ?

<your search> | rex "(?m)^(?<time>.*?)\s(?<collector_IP_Address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?<Type>.*)#(?<Alert_id>.*),\sstart\s(?<start_time>.*),\sduration\s(?<duration>\d+),\sdirection\s(?<direction>\w+),\shost\s(?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?<signatures>.*)\),\simpact\s(?<impact>.*),\simportance\s(?<importance>\d+),\smanaged_objects\s\(\"(?<managed_objects>\w+)\"\),\s\(parent\smanaged\sobject\s\"(?<parent_managed_object>\w+)\"\)"

For reference please check regex with sample data at https://regex101.com/r/IOZJxI/2/

I hope this helps.

Thanks,
Harshil

View solution in original post

0 Karma

New Member

Thank you so much Harshil. Let me show you the query I am entering and maybe you can help me pinpoint my error.

index=tonga_logs host=192.168.69.50 "Host Detection" 
| rex "(?m)^(?.*?)\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?.*)#(?.*),\sstart\s(?.*),\sduration\s(?\d+),\sdirection\s(?\w+),\shost\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?.*)\),\simpact\s(?.*),\simportance\s(?\d+),\smanaged_objects\s\(\"(?\w+)\"\),\s\(parent\smanaged\sobject\s\"(?\w+)\"\)"
|table time, collector_IP_Address, Type, Alert_id, start_time, duration, direction, host, signatures, impact, importance, managed_objects

I feel like everything is there but it still doesn't display anything for me.

0 Karma

SplunkTrust
SplunkTrust

Can you please post your query and sample data in Code Sample format (Use Button 101010)

0 Karma

New Member

If you don't mind me asking: could you direct me how to do that?

0 Karma

Legend

When you start typing in the Text Box here on Splunk Answers you will see an icon 101010 which is the Code button. This will prevent special characters in your code from being escaped.

Alternatively, when you start typing in your SPL, you can prefix four spaces just before each new line of your code.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Community Manager
Community Manager

Thanks for sharing that tip with them @niketnilay 🙂 I've just gone ahead and reformatted it in a Code Sample box already so all special characters are visible.

0 Karma