Splunk Search
Highlighted

How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Explorer

I currently have a log statement which has a custom delimiter: {|}

Where an example log statement would look like:

Oct-13 12:17:13 | INFO| [Logger:152] Message{|}Activity1{|}userDeletedProfile{|}John Smith{|}Smith Securities{|}Test1{|}5512{|}324166{|}552341{|}260

However, when I try to conduct a field extraction where DELIMS = "{|}", the fields aren't being extracted properly. However, testing the above log statement in another application that is capable of delimiting yields successful results.

0 Karma
Highlighted

Re: How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Champion

The community probably needs more info to help. Can you share more details about the settings in props.conf that are associated with this sourcetype?

0 Karma
Highlighted

Re: How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Champion

I think you cannot use the DELIMS setting to use ordered groups of characters as delimeters. Reading http://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf seems to say that you can only specify a single character delimeter.

Instead of DELIM you will probably have to specify the fields/delimeters with REGEX-based extractions.

Highlighted

Re: How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Legend

You have to extract fields using multiple regex.
I cannot use my pc now, Tomorrow morning I'll send you an example.
Bye.
Giuseppe

0 Karma
Highlighted

Re: How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Legend

in props. conf
REPORT-myfields = myfields_Fields

in transforms.conf
[myfieldsFields]
REGEX = ](.){|}(.){|}(.){|}(.){|}(.){|}(.){|}(.){|}(.){|}(.){|}(.)
FORMAT = Field1::"$1" Field2::"$2" Field3::"$3" Field4::"$4" Field5::"$5" Field6::"$6" Field7::"$7" Field8::"$8" Field9::"$9" Field10::"$10"
WRITE
META = true

Bye.
Giuseppe

0 Karma
Highlighted

Re: How do I configure Splunk to recognize my custom delimiter for proper field extraction?

Motivator

Hi @emamedov

I tried to extract fields using the Field Extractor and it did extract the fields with {|} however for some odd reason it considered pipe in between as a field too. So all of below in " " are each field as they appear on my field extractor.

"Oct-13 12:18:13 | INFO| [Logger:152] Message"  "|"  "Activity1" "|" "userDeletedProfile" "|" "John" "|" "KMI SECUR" "|" "Test1" "|" "5512" "|" "324166" "|" "552341" "|" "260"

If you are not fine with above then alternatively you can use below regex in "write my own regular expression" in FieldExtractor or during search time:


^(?< dateTime>[^|]+)|\s+?(?< loggerLevel>[^|]+)|\s+?(?< message>[^{]+){|}(?< activity>[^{]+){|}(?< profile>[^{]+){|}(?< userName>[^{]+){|}(?< securities>[^{]+){|}(?< test>[^{]+){|}(?< id1>[^{]+){|}(?< id2>[^{]+){|}(?< id3>[^{]+){|}(?< id4>[^{]+)

Note Please remove space in each of the tags above < dateTime> < loggerLevel> and so on

0 Karma