Splunk Search

How to configure props and transforms to parse pipe separated fields from my log file with 3 different event patterns?

New Member

I have an alert dump coming from one of our tools and it contains events in the following format. However, there are three different patterns. I am unable to parse them properly and perform field extraction. the fields are separated by a | symbol and are in the format specified below:

Event Format-

  MsgID|DateTime|MessageType|Icon|Message|ObjectType|ObjectID|ObjectID2|IPAddress|Caption|BackColor|Acknowledged|ActiveNetObject|NetObjectPrefix

1) Event pattern 1

401683|2015-06-08 18:44:58.433000000|event|5000|Interface xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1 for node xyz.companyname.co.uk has a transmitted utilization of 76 which is greater than the threshold of 75%.|I         |1708||10.47.106.68||12648447|0|1708|I         

2) Event Pattern 2

3B36E06E-0F36-4DB0-B5A7-BD310EC217EC|2015-06-08 18:44:58.380000000|advanced alert|0|High Transmit Percent Utilization|Interface|1708|0|10.47.106.68|xyz.companyname.co.uk - GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I

3) Event Pattern 3

30106255|2015-06-08 18:39:32.033000000|trap|0|netscreenTrapDesc=2015-06-08 18:39:31 [Root]system-critical-00040: VPN 'NY_Tunnel' from 208.105.9.106 is up.  

netscreenTrapType=vpn-tunnel-up(40)

snmpTrapOID=NETSCREEN-TRAP-MIB:netscreenTrapVpn

sysUpTime=14 days 0 hours 1 minute 23.00 seconds

|N|149|0|10.67.1.18 |10.67.1.18|16777215|0|149|N

401675|2015-06-08 18:17:12.253000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I         |1792||10.47.106.68||12648447|0|1792|I         
8|2015-06-08 18:17:12.240000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABCis 54.
Current traffic load of this interface is 
     Received : 2.70 M
     Transmitted : 514 M|I|1792|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/4/33 · Trunk to ABC|0|0|1792|I
8|2015-06-08 18:17:12.177000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I|1711|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4|0|0|1711|I
401674|2015-06-08 18:17:12.173000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/4 - Gi1/1/4 is 67.
Current traffic load of this interface is 
     Received : 513 M
     Transmitted : 637 M|I         |1711||10.47.106.68||12648447|0|1711|I         
401673|2015-06-08 18:17:12.143000000|event|5000|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I         |1708||10.47.106.68||12648447|0|1708|I         
8|2015-06-08 18:17:11.740000000|basic alert|0|Alert: Transmit Percent Utilization of xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1 is 51.
Current traffic load of this interface is 
     Received : 123 M
     Transmitted : 487 M|I|1708|0|10.47.106.68|xyz.companyname.co.uk-GigabitEthernet1/1/1 - Gi1/1/1|0|0|1708|I
0 Karma

Path Finder

Not sure if I completely understand the question b/c you say there are 3 different patterns but then you give the format you're trying to capture. But give this a shot....

Try using a props/transforms combo like this:

props.conf

[sourcetypeName]
REPORT-getData = getLogData

transforms.conf

[getLogData]
DELIMS = "|"
FIELDS = MsgID, DateTime, MessageType, Icon, Message, ObjectType, ObjectID, ObjectID2, IPAddress, Caption, BackColor, Acknowledged, ActiveNetObject, NetObjectPrefix

You can then search by each name in the 'FIELDS' section and I believe you'll be good-to-go.

0 Karma

New Member

Could not set the correct line breaker in my props.conf to extract the fields properly.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!