topic Re: Inconsistent Delimited Values in Splunk Search

Inconsistent Delimited Values

lloydknight — Mon, 06 Feb 2017 12:29:07 GMT

Here's my search:

 base search | rex "^(?<field1>[^:]+):\|:(?<field2>[^:]+):\|:(?<field3>[^:]+):\|:"

The logs are delimited by :|: and this search about works fine but some events have inconsistent values with 6 or 9 or 12 fields but 97% of the logs have 3 fields only. This 6 or 9 or 12 fields are 2 or 3 or 4 events that appear in 1 line.

Extraction should be done in search only.

Anyone here encountered this kind of logs?

Any input would be very much appreciated.

Thanks.

EDIT:

Sample Logs

2/1/17
11:59:59.000 PM 
396629369  396629369:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
665341651  665341651:|:0:|:U:|:
665341657  665341657:|:0:|:U:|:
665341660  665341660:|:0:|:E:|:
665341654  665341654:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629368:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
665341651  665341652:|:0:|:U:|:
665341657  665341653:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629366:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629367:|:0:|:U:|:

Re: Inconsistent Delimited Values

gcusello — Mon, 06 Feb 2017 13:09:14 GMT

can you share an example of your logs?
Bye.
Giuseppe

Re: Inconsistent Delimited Values

lloydknight — Mon, 06 Feb 2017 13:21:03 GMT

please see edit for the sample logs

Re: Inconsistent Delimited Values

lloydknight — Mon, 06 Feb 2017 14:32:22 GMT

Hello.

Already solved this.

base search | rex "^((?<field1>[^:]+):\|:(?<field2>[^:]+):\|:(?<field3>[^:]+):)?(\|:(?<field4>[^:]+):\|:(?<field5>[^:]+):\|:(?<field6>[^:]+):)?(\|:(?<field7>[^:]+):\|:(?<field8>[^:]+):\|:(?<field9>[^:]+):)?(\|:(?<field10>[^:]+):\|:(?<field11>[^:]+):\|:(?<field12>[^:]+):\|:)?"
| eval newfield1=mvappend(field1, field4, field7, field10)
| eval newfield2=mvappend(field2, field5, field8, field11)
| eval newfield3=mvappend(field3, field6, field9, field12)

Grouped them into 3 then used "?"
and used mvappend for the newfield

Thank you.