Splunk Search
Highlighted

How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

Explorer

Hello,

I have Message-Tracking Logs from Exchange 2016 servers where the fields are comma separated, but in some lines Microsoft uses Sub-Fields. In this case the Main field is quoted and a commas are used again as separators inside this main field.

Example:

2016-12-20T14:33:54.693Z,fe80::b9c4:56fa:d460:81f3,exchangesrv.test.com,fe80::b9c4:56fa:d460:81f3%12,exchangesrv,"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD",,STOREDRIVER,RECEIVE,2014,,4c7b2b2e-291a-4635-f603-08d428e539b5,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,To,8944,1,,,0000003a-0000-0000-0000-0000f519e953-MBTSubmissionServiceHeartbeatProbe,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,04I: ,Originating,,,,S:MailboxDatabaseGuid=bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d;S:ItemEntryId=00-00-00-00-2D-E8-87-0C-DF-D8-A0-42-97-64-3D-D7-57-8C-2A-3C-07-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-00-00-01-0B-00-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-1E-84-2C-F6-00-00;S:DeliveryPriority=Normal;S:AccountForest=test.com;S:IsProbe=true;S:PersistProbeTrace=False,Email,940aee54-2531-41e4-f603-08d428e539b5,15.01.0544.027

This part below should be just one field:

"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD"

The problem now is that Splunk (6.4.2) does not handle the quoted field as one field - it splits it up to 6 separate fields. Can I change that behavior? I found the following question where Splunk does the magic I want and interprets everything within the quotes as one field:
https://answers.splunk.com/answers/99398/delims-fields-with-a-field-that-has-sub-fields.html?utm_sou...

Is there an option in transforms.conf I am missing?

Thanks,
/mspoerr

0 Karma
Highlighted

Re: How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

Builder

A quick workaround should be to extract the quoted field separately from the _raw data. You can do it through field extractions or props.conf and transforms.conf. Here I would give you the example using inline rex command:

-- Your query -- | eval _raw = " 2016-12-20T14:33:54.693Z,fe80::b9c4:56fa:d460:81f3,exchangesrv.test.com,fe80::b9c4:56fa:d460:81f3%12,exchangesrv,\"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD\",,STOREDRIVER,RECEIVE,2014,,4c7b2b2e-291a-4635-f603-08d428e539b5,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,To,8944,1,,,0000003a-0000-0000-0000-0000f519e953-MBTSubmissionServiceHeartbeatProbe,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,04I: ,Originating,,,,S:MailboxDatabaseGuid=bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d;S:ItemEntryId=00-00-00-00-2D-E8-87-0C-DF-D8-A0-42-97-64-3D-D7-57-8C-2A-3C-07-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-00-00-01-0B-00-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-1E-84-2C-F6-00-00;S:DeliveryPriority=Normal;S:AccountForest=test.com;S:IsProbe=true;S:PersistProbeTrace=False,Email,940aee54-2531-41e4-f603-08d428e539b5,15.01.0544.027" | rex field=_raw "\"(?<quoted>.*)\"" | table _raw quoted

0 Karma
Highlighted

Re: How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

Explorer

Thanks for your comment but it seems I wasn't specific enough. I would like to extract all 30 fields not just the special one.

0 Karma
Highlighted

Re: How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

Builder

There may be a better solution but as a workaround I think use automatically extracted fields for all other fields and extract the field in quotes separately from raw data.

0 Karma
Highlighted

Re: How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

Explorer

In a second try it suddenly works. I am not sure if I just was impatient or I have overseen something when I configured it the first time...

0 Karma