How to break events on Particular field using Regex or any other process?

[mprreddy51]

Hi All,

Below is my event data:

Issue 1:

11/11/15 
1:26:01.000 PM  
Job Id, Class Id,"Id","Success","Created","Error","Id","Service_Team_Members_Initials__c"
950210000002JwhAAE,651270000007OwIAAU,"","false","false","FIELD_CUSTOM_VALIDATION_EXCEPTION:01:  The selected Contact Role is not valid for this type of Office. Valid option is RIA:Contact_Role_and_Registration__c --","006C000001NM6TpIAL","ABCD, JERW, XCVB, CGLC, JSWB, BCH, LAAM, TCWJ, DAFC, KJJH, SCAH, BIMR"
650240000003JwhAAE,451270000007OwIAAU,"","false","false","FIELD_CUSTOM_VALIDATION_EXCEPTION:01:  The selected Contact Role is not valid for this type of Office. Valid option is RIA:Contact_Role_and_Registration__c --","005C000001Se9x2IAB","EFGH, TWEW, SDFR, MCQ, JOLP, MPK, SCRC, LAAM, LAAM, JNAC, SCAH, JSDF, CDER, DAFC, KJJH, BCH"

1) I want to skip headers jobid,classid,"ID","Success","Error","Id","Service_Team_Members_Initials__c" in the events.
2) I need to know how to break the events on Job Id, and after breaking all events, should have the same date and time by using regex or any other method to solve this.

Issue 2:

11/10/15 
5:48:13.000 AM  
1-0000642980,,,,,290641,Sent WMS,,Proposal Order,Error received from salesesssd.com.  Fields [Requested_Delivery_Date__c].  Status code [FIELD_CUSTOM_VALIDATION_EXCEPTION].  Message [Please select Requested Delivery Date].
1-0000642131,,,,,290480,Sent WMS,,Proposal Order,Error received from salesesssd.com.  Fields [Requested_Delivery_Date__c].  Status code [FIELD_CUSTOM_VALIDATION_EXCEPTION].  Message [Please select Requested Delivery Date].
1-0000642138,,,,,290485,Sent WMS,,Proposal Order,Error received from salesesssd.com.  Fields [Requested_Delivery_Date__c].  Status code [FIELD_CUSTOM_VALIDATION_EXCEPTION].  Message [Please select Requested Delivery Date].

1) I want to break these events on the newline character.

Thanks in advance, and any method to make this work is fine.
I used this for issue2: LINE_BREAKER = ([\r\n]+) in props.conf

Thanks in advance.

[jmallorquin]

Hi,

For issue 1 to erase the header
in props.conf
[YOURSOURCETYPE]
TRANSFORMS-delete-header = eliminate-header

in transforms.conf
[eliminate-header]

REGEX=^Job\s+Id

DEST_KEY=queue
FORMAT=nullQueue

For issue 2

LINE_BREAKER = ([\r\n]+)\d\-\d+\,)

Hope help you