Splunk Search

Need help with regex, mvindex, or other option for field-extractions

avishek_08
New Member
  1. Dec 5 18:04:51 192.168.69.50 pfsp: Host Detection alert #22049413, start 2017-12-06 00:03:45 GMT, duration 66, direction incoming, host 71.92.104.13, signatures (ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification), impact 4.00 Gbps/386.20 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")
  2. Dec 5 16:19:51 192.168.69.50 pfsp: Host Detection alert #22049331, start 2017-12-05 22:16:45 GMT, duration 186, direction incoming, host 24.177.66.30, signatures (ICMP, IP Fragmentation, TCP NULL, TCP SYN, TCP RST, Total Traffic, UDP), impact 310.64 Mbps/104.45 Kpps, importance 2, managed_objects ("Tonga"), (parent managed object "nil")

Please help me parse this log event. For some reason, it has been parsed as a single field in my Splunk instance. On top of it all, I do not have access to .prof files to change the field extraction criteria hence need to utilize Splunk commands to properly parse them to create a table.
Please help me parse the log as following>> _time: Dec 5 18:04:51, collector IP_address: 192.168.69.50, Type: Host Detection, Alert_id: 22049413, start_time: 2017-12-06 00:03:45 GMT, duration: 66, direction: incoming, host: 71.92.104.13, signatures: ICMP, IP Fragmentation, Total Traffic, UDP, DNS Amplification, impact: 4.00 Gbps/386.20 Kpps, importance: 2, managed_objects: Tonga, parent managed object: nil.

I have been trying to use Rex commands but since I am not an expert at it. Its not working out so much. Colleague suggested mvindex, I am trying to parse it using that but I hope I can get some help from awesome Splunk community.

0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Hi @avishek_08,

Will you please try this regex ?

<your search> | rex "(?m)^(?<time>.*?)\s(?<collector_IP_Address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?<Type>.*)#(?<Alert_id>.*),\sstart\s(?<start_time>.*),\sduration\s(?<duration>\d+),\sdirection\s(?<direction>\w+),\shost\s(?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?<signatures>.*)\),\simpact\s(?<impact>.*),\simportance\s(?<importance>\d+),\smanaged_objects\s\(\"(?<managed_objects>\w+)\"\),\s\(parent\smanaged\sobject\s\"(?<parent_managed_object>\w+)\"\)"

For reference please check regex with sample data at https://regex101.com/r/IOZJxI/2/

I hope this helps.

Thanks,
Harshil

View solution in original post

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @avishek_08,

Will you please try this regex ?

<your search> | rex "(?m)^(?<time>.*?)\s(?<collector_IP_Address>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?<Type>.*)#(?<Alert_id>.*),\sstart\s(?<start_time>.*),\sduration\s(?<duration>\d+),\sdirection\s(?<direction>\w+),\shost\s(?<host>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?<signatures>.*)\),\simpact\s(?<impact>.*),\simportance\s(?<importance>\d+),\smanaged_objects\s\(\"(?<managed_objects>\w+)\"\),\s\(parent\smanaged\sobject\s\"(?<parent_managed_object>\w+)\"\)"

For reference please check regex with sample data at https://regex101.com/r/IOZJxI/2/

I hope this helps.

Thanks,
Harshil

0 Karma

avishek_08
New Member

Thank you so much Harshil. Let me show you the query I am entering and maybe you can help me pinpoint my error.

index=tonga_logs host=192.168.69.50 "Host Detection" 
| rex "(?m)^(?.*?)\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\w+\:\s(?.*)#(?.*),\sstart\s(?.*),\sduration\s(?\d+),\sdirection\s(?\w+),\shost\s(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),\ssignatures\s\((?.*)\),\simpact\s(?.*),\simportance\s(?\d+),\smanaged_objects\s\(\"(?\w+)\"\),\s\(parent\smanaged\sobject\s\"(?\w+)\"\)"
|table time, collector_IP_Address, Type, Alert_id, start_time, duration, direction, host, signatures, impact, importance, managed_objects

I feel like everything is there but it still doesn't display anything for me.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Can you please post your query and sample data in Code Sample format (Use Button 101010)

0 Karma

avishek_08
New Member

If you don't mind me asking: could you direct me how to do that?

0 Karma

niketn
Legend

When you start typing in the Text Box here on Splunk Answers you will see an icon 101010 which is the Code button. This will prevent special characters in your code from being escaped.

Alternatively, when you start typing in your SPL, you can prefix four spaces just before each new line of your code.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

ppablo
Retired

Thanks for sharing that tip with them @niketnilay 🙂 I've just gone ahead and reformatted it in a Code Sample box already so all special characters are visible.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...