Splunk Dev

How to extract fields in Heavy Forwarder?

R_M
Loves-to-Learn

Guys if you help me to extract fields from the raw events in props.conf in HF, I tried  EXTRACT command seems my regex is not ok or not sure what is the issue.

I want to extract field and give name to them. 

Regex I tried:

^(?:[^,\n]*,){7}(?<src_ip>[^,]+),(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+),(?<dst_port>\d+)(?:[^,\n]*,){5}(?<action>[^,]+)(?:[^,\n]*,){38}

Also,

^(?:[^,\n]*,){7}src_ip=(?<src_ip>[^,]+),dst_ip=(?<dst_ip>[^,]+)(?:[^:\n]*:){2}\d+,\d+,\d+,src_port=(?<src_port>\d+),dst_port=(?<dst_port>\d+)(?:[^,\n]*,){5}action=(?<action>[^,]+)(?:[^,\n]*,){38}

Sample log: 

Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,,

Please help me to extract fields like src_ip, dst_ip, src_port, dst_port, action etc.

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Wait a minute. EXTRACT is for search-time extractions. You don't use it on HF since you don't search on HF.

0 Karma

mayurr98
Super Champion

Hey could you please try this :

 

| makeresults 
| eval _raw="Mar 31 18:18:35 LUM-EVERE-PAFW-R8-17-T1 1,2022/03/31 18:18:35,015701001564,TRAFFIC,drop,2305,2022/03/31 18:18:35,10.81.13.68,34.240.162.53,0.0.0.0,0.0.0.0,prodedfl_access_1289,,,not-applicable,vsys4,prodedfl,prodcore,ae1.1512,,Syslog_Server,2022/03/31 18:18:35,0,1,60353,443,0,0,0x0,tcp,deny,66,66,0,1,2022/03/31 18:18:35,0,any,0,7022483376390954281,0x8000000000000000,10.0.0.0-10.255.255.255,Ireland,0,1,0,policy-deny,920,0,0,0,Production,LUM-EVERE-PAFW-R8-17-T1,from-policy,,,0,,0,,N/A,0,0,0,0,2d8c02f8-e86f-43cf-a459-01acdb26580a,0,0,,,,,,," 
| rex "\d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)"

 

let me know if this helps! 

 

Thanks,

Mayur 

0 Karma

R_M
Loves-to-Learn

Can I apply this regex in props.conf under EXTRACT commmand, also  I also want to give name to the field, example: src_ip: 10.82.69.5

0 Karma

mayurr98
Super Champion

yes you can use this regex in props.conf. 

if you want to add a search time field extraction within props.conf, just use EXTRACT

 

[your-sourcetype]    
EXTRACT-<class> = [<regex>|<regex> in <src_field>]
    * Used to create extracted fields (search-time field extractions) that do
      not reference transforms.conf stanzas.

 

for reference see : http://docs.splunk.com/Documentation/Splunk/8.2.5/Admin/Propsconf

Please keep in mind that this will require a refresh/debug= http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh

 
 
0 Karma

R_M
Loves-to-Learn

Please help me how I can apply it, if you can share command

0 Karma

mayurr98
Super Champion

you need to use this regex on search head 

go to 

Settings » Fields » Field extractions » Add new

 

Destination App: <your_app>
Name: <name>
Apply to: choose sourcetype :  named <your_sourcetype>
Type: Inline
Extraction/Transform: \d{2}:\d{2}:\d{2},(?<src_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?<dst_ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))\,(?:[^:\n]*:){2}\d+,\d+,\d+,(?<src_port>\d+)\,(?<dst_port>[^\,]+)(?:[^,\n]*,){5}(?<action>[^\,]+)

 

let me know if this helps!

 

 
 
 
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is that a single event or multiple events?  

Please identify the fields within the log (which IP address is source/dest, etc.).

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...