How to extract fields in Heavy Forwarder?

[R_M]

Guys if you help me to extract fields from the raw events in props.conf in HF, I tried  EXTRACT command seems my regex is not ok or not sure what is the issue.

I want to extract field and give name to them. 

Regex I tried:

^(?:[^,\n]*,){7}(?<src_ip>[^,]+),(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+),(?<dst_port>\d+)(?:[^,\n]*,){5}(?<action>[^,]+)(?:[^,\n]*,){38}

Also,

^(?:[^,\n]*,){7}src_ip=(?<src_ip>[^,]+),dst_ip=(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,src_port=(?<src_port>\d+),dst_port=(?<dst_port>\d+)(?:[^,\n]*,){5}action=(?<action>[^,]+)(?:[^,\n]*,){38}

Sample log: 

Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,,

Please help me to extract fields like src_ip, dst_ip, src_port, dst_port, action etc.

[PickleRick]

Wait a minute. EXTRACT is for search-time extractions. You don't use it on HF since you don't search on HF.

[mayurr98]

Hey could you please try this :

| makeresults 
| eval _raw="Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,," 
| rex "\d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)"

let me know if this helps! 

Thanks,

Mayur 

[R_M]

Can I apply this regex in props.conf under EXTRACT commmand, also  I also want to give name to the field, example: src_ip: 10.82.69.5

[mayurr98]

yes you can use this regex in props.conf. 

if you want to add a search time field extraction within props.conf, just use EXTRACT

[your-sourcetype]    
EXTRACT-<class> = [<regex>|<regex> in <src_field>]
    * Used to create extracted fields (search-time field extractions) that do
      not reference transforms.conf stanzas.

for reference see : http://docs.splunk.com/Documentation/Splunk/8.2.5/Admin/Propsconf

Please keep in mind that this will require a refresh/debug= http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh

[R_M]

Please help me how I can apply it, if you can share command

[mayurr98]

you need to use this regex on search head 

go to 

Settings » Fields » Field extractions » Add new

Destination App: <your_app>
Name: <name>
Apply to: choose sourcetype :  named <your_sourcetype>
Type: Inline
Extraction/Transform: \d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)

let me know if this helps!

[richgalloway]

Is that a single event or multiple events?  

Please identify the fields within the log (which IP address is source/dest, etc.).