Splunk Dev

How to extract fields in Heavy Forwarder?

R_M
Loves-to-Learn

Guys if you help me to extract fields from the raw events in props.conf in HF, I tried  EXTRACT command seems my regex is not ok or not sure what is the issue.

I want to extract field and give name to them. 

Regex I tried:

^(?:[^,\n]*,){7}(?<src_ip>[^,]+),(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+),(?<dst_port>\d+)(?:[^,\n]*,){5}(?<action>[^,]+)(?:[^,\n]*,){38}

Also,

^(?:[^,\n]*,){7}src_ip=(?<src_ip>[^,]+),dst_ip=(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,src_port=(?<src_port>\d+),dst_port=(?<dst_port>\d+)(?:[^,\n]*,){5}action=(?<action>[^,]+)(?:[^,\n]*,){38}

Sample log: 

Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,,

Please help me to extract fields like src_ip, dst_ip, src_port, dst_port, action etc.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Wait a minute. EXTRACT is for search-time extractions. You don't use it on HF since you don't search on HF.

0 Karma

mayurr98
Super Champion

Hey could you please try this :

 

| makeresults 
| eval _raw="Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,," 
| rex "\d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)"

 

let me know if this helps! 

 

Thanks,

Mayur 

0 Karma

R_M
Loves-to-Learn

Can I apply this regex in props.conf under EXTRACT commmand, also  I also want to give name to the field, example: src_ip: 10.82.69.5

0 Karma

mayurr98
Super Champion

yes you can use this regex in props.conf. 

if you want to add a search time field extraction within props.conf, just use EXTRACT

 

[your-sourcetype]    
EXTRACT-<class> = [<regex>|<regex> in <src_field>]
    * Used to create extracted fields (search-time field extractions) that do
      not reference transforms.conf stanzas.

 

for reference see : http://docs.splunk.com/Documentation/Splunk/8.2.5/Admin/Propsconf

Please keep in mind that this will require a refresh/debug= http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh

 
 
0 Karma

R_M
Loves-to-Learn

Please help me how I can apply it, if you can share command

0 Karma

mayurr98
Super Champion

you need to use this regex on search head 

go to 

Settings » Fields » Field extractions » Add new

 

Destination App: <your_app>
Name: <name>
Apply to: choose sourcetype :  named <your_sourcetype>
Type: Inline
Extraction/Transform: \d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)

 

let me know if this helps!

 

 
 
 
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is that a single event or multiple events?  

Please identify the fields within the log (which IP address is source/dest, etc.).

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...