Configuring Transforms.conf for log with multiple delimiters

ezajac — Tue, 23 Oct 2012 12:43:24 GMT

I have a new log that I need to define a sourcetype for. There are several different ways the fields are delimited.

The formatting is defined in a Log4Net.config file is below. Is there an easy way to configure this in the transforms.conf file?

2012-10-17 12:06:43,546 [53] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyData203.125
2012-10-17 12:06:43,656 [6] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyData203.125
2012-10-17 12:06:43,765 [53] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyNameAddress156.25
2012-10-17 12:06:43,828 [53] INFO HTTPPages [(null)] - PROD\AOSHOGAYHTTPPagehttp://plcontactview.prodlb.travp.net/plcontactview/PLContactSearch.aspx?txtClientFunction=InsuredSearch&ixn_subtype=null&ixnID=02420216ac9589cd&ixn_type=null&txtBAOrPol=9872514641011671.875

GET
2012-10-17 12:06:43,875 [6] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyNameAddress203.125
2012-10-17 12:06:43,921 [6] INFO HTTPPages [(null)] - PROD\BROBERT4HTTPPagehttp://plcontactview.prodlb.travp.net/plcontactview/PLContactSearch.aspx?txtClientFunction=PhoneSearch&ixn_subtype=null&ixnID=01d00216ab58c5e0&ixn_type=null&txtPhone=8047526988640.625

398POST
2012-10-17 12:06:44,234 [6] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyData250
2012-10-17 12:06:44,265 [41] INFO ServiceCalls [(null)] - ServiceCallPreferenceManagementServiceHook.GetPreferences218.75
2012-10-17 12:06:44,328 [16] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyData234.375
2012-10-17 12:06:44,328 [53] INFO ServiceCalls [(null)] - ServiceCallPreferenceManagementServiceHook.GetPreferences156.25
2012-10-17 12:06:44,421 [41] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetClaimData156.25
2012-10-17 12:06:44,437 [41] INFO HTTPPages [(null)] - PROD\BROBERT4HTTPPagehttp://plcontactview.prodlb.travp.net/plcontactview/PLBillingTreeView.aspx421.875

GET
2012-10-17 12:06:44,437 [6] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyNameAddress171.875
2012-10-17 12:06:44,531 [41] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetProducerData46.875
2012-10-17 12:06:44,531 [53] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetClaimData203.125
2012-10-17 12:06:44,546 [16] INFO ServiceCalls [(null)] - ServiceCallCRDBServiceHook.GetPolicyNameAddress187.5

Re: Configuring Transforms.conf for log with multiple delimiters

davecroto — Wed, 24 Oct 2012 02:54:13 GMT

Set a common, single sourcetype for the 2 type of events you show above. Use a tranforms and props stanza like I did here... I called my sourcetype log4net. Now, all the events from that source with either the word ServiceCalls or HTTPPages will be sourcetyped as log4net. Then just refer to the new sourcetype in the props.conf file for the field extractions.

tranforms.conf

[log4net]
REGEX = (ServiceCalls|HTTPPages)
FORMAT = sourcetype::log4net
DEST_KEY = MetaData:Sourcetype

props.conf

[source::…/bla/bla/log4nettype.log]
TRANSFORMS-log4net =log4net

[log4net]
EXTRACT-log4net = (?i)\[(?\d+)\]\s+(?\w+)\s+(?\w+)\s+\[\((?\w+)\)\]\s+\-(?.*)

Re: Configuring Transforms.conf for log with multiple delimiters

davecroto — Thu, 25 Oct 2012 11:13:44 GMT

Noted that the "backslashes" in the Extract needed to be escaped to show up on this post. Let me know if this worked for you....

topic Re: Configuring Transforms.conf for log with multiple delimiters in Getting Data In

Configuring Transforms.conf for log with multiple delimiters

Re: Configuring Transforms.conf for log with multiple delimiters

Re: Configuring Transforms.conf for log with multiple delimiters