Inconsistent Delimited Values

[lloydknight]

Here's my search:

 base search | rex "^(?<field1>[^:]+):\|:(?<field2>[^:]+):\|:(?<field3>[^:]+):\|:"

The logs are delimited by :|: and this search about works fine but some events have inconsistent values with 6 or 9 or 12 fields but 97% of the logs have 3 fields only. This 6 or 9 or 12 fields are 2 or 3 or 4 events that appear in 1 line.

Extraction should be done in search only.

Anyone here encountered this kind of logs?

Any input would be very much appreciated.

Thanks.

EDIT:

Sample Logs

2/1/17
11:59:59.000 PM 
396629369  396629369:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
665341651  665341651:|:0:|:U:|:
665341657  665341657:|:0:|:U:|:
665341660  665341660:|:0:|:E:|:
665341654  665341654:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629368:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
665341651  665341652:|:0:|:U:|:
665341657  665341653:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629366:|:0:|:U:|:
2/1/17
11:59:59.000 PM 
396629369  396629367:|:0:|:U:|:

[lloydknight · Accepted solution]

Hello.

Already solved this.

base search | rex "^((?<field1>[^:]+):\|:(?<field2>[^:]+):\|:(?<field3>[^:]+):)?(\|:(?<field4>[^:]+):\|:(?<field5>[^:]+):\|:(?<field6>[^:]+):)?(\|:(?<field7>[^:]+):\|:(?<field8>[^:]+):\|:(?<field9>[^:]+):)?(\|:(?<field10>[^:]+):\|:(?<field11>[^:]+):\|:(?<field12>[^:]+):\|:)?"
| eval newfield1=mvappend(field1, field4, field7, field10)
| eval newfield2=mvappend(field2, field5, field8, field11)
| eval newfield3=mvappend(field3, field6, field9, field12)

Grouped them into 3 then used "?"
and used mvappend for the newfield

Thank you.

[gcusello]

can you share an example of your logs?
Bye.
Giuseppe

[lloydknight]

please see edit for the sample logs