Building for the Splunk Platform

How to extract fields in Heavy Forwarder?

R_M
Loves-to-Learn

Guys if you help me to extract fields from the raw events in props.conf in HF, I tried  EXTRACT command seems my regex is not ok or not sure what is the issue.

I want to extract field and give name to them. 

Regex I tried:

^(?:[^,\n]*,){7}(?<src_ip>[^,]+),(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+),(?<dst_port>\d+)(?:[^,\n]*,){5}(?<action>[^,]+)(?:[^,\n]*,){38}

Also,

^(?:[^,\n]*,){7}src_ip=(?<src_ip>[^,]+),dst_ip=(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,src_port=(?<src_port>\d+),dst_port=(?<dst_port>\d+)(?:[^,\n]*,){5}action=(?<action>[^,]+)(?:[^,\n]*,){38}

Sample log: 

Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,,

Please help me to extract fields like src_ip, dst_ip, src_port, dst_port, action etc.

Labels (1)
0 Karma

PickleRick
Ultra Champion

Wait a minute. EXTRACT is for search-time extractions. You don't use it on HF since you don't search on HF.

0 Karma

mayurr98
Super Champion

Hey could you please try this :

 

| makeresults 
| eval _raw="Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,," 
| rex "\d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)"

 

let me know if this helps! 

 

Thanks,

Mayur 

0 Karma

R_M
Loves-to-Learn

Can I apply this regex in props.conf under EXTRACT commmand, also  I also want to give name to the field, example: src_ip: 10.82.69.5

0 Karma

mayurr98
Super Champion

yes you can use this regex in props.conf. 

if you want to add a search time field extraction within props.conf, just use EXTRACT

 

[your-sourcetype]    
EXTRACT-<class> = [<regex>|<regex> in <src_field>]
    * Used to create extracted fields (search-time field extractions) that do
      not reference transforms.conf stanzas.

 

for reference see : http://docs.splunk.com/Documentation/Splunk/8.2.5/Admin/Propsconf

Please keep in mind that this will require a refresh/debug= http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh

 
 
0 Karma

R_M
Loves-to-Learn

Please help me how I can apply it, if you can share command

0 Karma

mayurr98
Super Champion

you need to use this regex on search head 

go to 

Settings » Fields » Field extractions » Add new

 

Destination App: <your_app>
Name: <name>
Apply to: choose sourcetype :  named <your_sourcetype>
Type: Inline
Extraction/Transform: \d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)

 

let me know if this helps!

 

 
 
 
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is that a single event or multiple events?  

Please identify the fields within the log (which IP address is source/dest, etc.).

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

This Week's Community Digest - Splunk Community Happenings [9.26.22]

Get the latest news and updates from the Splunk Community here! Upcoming User Group Events! &#x1f44f; Check ...

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...