How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

kiranmudunuru — Thu, 18 Jun 2015 21:30:38 GMT

I have an alert dump coming from one of our tools and it contains events in the following format. However, there are three different patterns. I am unable to parse them properly and perform field extraction. the fields are separated by a | symbol and are in the format specified below:

Event Format-

  MsgID|DateTime|MessageType|Icon|Message|ObjectType|ObjectID|ObjectID2|IPAddress|Caption|BackColor|Acknowledged|ActiveNetObject|NetObjectPrefix

1) Event pattern 1

401683|2015-06-08 18:44:58.433000000|event|5000|Interface xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1 for node xyz.companyname.co.uk has a transmitted utilization of 76 which is greater than the threshold of 75%.|I         |1708||10.47.106.68||12648447|0|1708|I

2) Event Pattern 2

3B36E06E-0F36-4DB0-B5A7-BD310EC217EC|2015-06-08 18:44:58.380000000|advanced alert|0|High Transmit Percent Utilization|Interface|1708|0|10.47.106.68|xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I

3) Event Pattern 3

30106255|2015-06-08 18:39:32.033000000|trap|0|netscreenTrapDesc=2015-06-08 18:39:31 [Root]system-critical-00040: VPN 'NY_Tunnel' from 208.105.9.106 is up.

netscreenTrapType=vpn-tunnel-up(40)

snmpTrapOID=NETSCREEN-TRAP-MIB:netscreenTrapVpn

sysUpTime=14 days 0 hours 1 minute 23.00 seconds

|N|149|0|10.67.1.18 |10.67.1.18|16777215|0|149|N

401675|2015-06-08 18:17:12.253000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I         |1792||10.47.106.68||12648447|0|1792|I         
8|2015-06-08 18:17:12.240000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I|1792|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABC|0|0|1792|I
8|2015-06-08 18:17:12.177000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I|1711|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4|0|0|1711|I
401674|2015-06-08 18:17:12.173000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I         |1711||10.47.106.68||12648447|0|1711|I         
401673|2015-06-08 18:17:12.143000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I         |1708||10.47.106.68||12648447|0|1708|I         
8|2015-06-08 18:17:11.740000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I|1708|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I

Re: How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

kiranmudunuru — Thu, 18 Jun 2015 21:32:34 GMT

Could not set the correct line breaker in my props.conf to extract the fields properly.

Re: How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

hogan24 — Thu, 18 Jun 2015 21:43:16 GMT

Not sure if I completely understand the question b/c you say there are 3 different patterns but then you give the format you're trying to capture. But give this a shot....

Try using a props/transforms combo like this:

props.conf

[sourcetypeName]
REPORT-getData = getLogData

transforms.conf

[getLogData]
DELIMS = "|"
FIELDS = MsgID, DateTime, MessageType, Icon, Message, ObjectType, ObjectID, ObjectID2, IPAddress, Caption, BackColor, Acknowledged, ActiveNetObject, NetObjectPrefix

You can then search by each name in the 'FIELDS' section and I believe you'll be good-to-go.

topic How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns? in Splunk Search

How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

Re: How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

Re: How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?