Splunk Search

How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

kiranmudunuru
New Member

I have an alert dump coming from one of our tools and it contains events in the following format. However, there are three different patterns. I am unable to parse them properly and perform field extraction. the fields are separated by a | symbol and are in the format specified below:

Event Format-

  MsgID|DateTime|MessageType|Icon|Message|ObjectType|ObjectID|ObjectID2|IPAddress|Caption|BackColor|Acknowledged|ActiveNetObject|NetObjectPrefix

1) Event pattern 1

401683|2015-06-08 18:44:58.433000000|event|5000|Interface xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1 for node xyz.companyname.co.uk has a transmitted utilization of 76 which is greater than the threshold of 75%.|I         |1708||10.47.106.68||12648447|0|1708|I         

2) Event Pattern 2

3B36E06E-0F36-4DB0-B5A7-BD310EC217EC|2015-06-08 18:44:58.380000000|advanced alert|0|High Transmit Percent Utilization|Interface|1708|0|10.47.106.68|xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I

3) Event Pattern 3

30106255|2015-06-08 18:39:32.033000000|trap|0|netscreenTrapDesc=2015-06-08 18:39:31 [Root]system-critical-00040: VPN 'NY_Tunnel' from 208.105.9.106 is up.  

netscreenTrapType=vpn-tunnel-up(40)

snmpTrapOID=NETSCREEN-TRAP-MIB:netscreenTrapVpn

sysUpTime=14 days 0 hours 1 minute 23.00 seconds

|N|149|0|10.67.1.18 |10.67.1.18|16777215|0|149|N

401675|2015-06-08 18:17:12.253000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I         |1792||10.47.106.68||12648447|0|1792|I         
8|2015-06-08 18:17:12.240000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I|1792|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABC|0|0|1792|I
8|2015-06-08 18:17:12.177000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I|1711|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4|0|0|1711|I
401674|2015-06-08 18:17:12.173000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I         |1711||10.47.106.68||12648447|0|1711|I         
401673|2015-06-08 18:17:12.143000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I         |1708||10.47.106.68||12648447|0|1708|I         
8|2015-06-08 18:17:11.740000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I|1708|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I
0 Karma

hogan24
Path Finder

Not sure if I completely understand the question b/c you say there are 3 different patterns but then you give the format you're trying to capture. But give this a shot....

Try using a props/transforms combo like this:

props.conf

[sourcetypeName]
REPORT-getData = getLogData

transforms.conf

[getLogData]
DELIMS = "|"
FIELDS = MsgID, DateTime, MessageType, Icon, Message, ObjectType, ObjectID, ObjectID2, IPAddress, Caption, BackColor, Acknowledged, ActiveNetObject, NetObjectPrefix

You can then search by each name in the 'FIELDS' section and I believe you'll be good-to-go.

0 Karma

kiranmudunuru
New Member

Could not set the correct line breaker in my props.conf to extract the fields properly.

0 Karma
Get Updates on the Splunk Community!

Set Up More Secure Configurations in Splunk Enterprise With Config Assist

This blog post is part 3 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...