Ingesting event data with timestamps afterwards sometimes

[Thomas-R]

Hey All,

I have data that needs to be ingested with multiple lines similar to the following:

************ Start Display Current Environment ************

***data***

***data***

***data***

************* End Display Current Environment *************

[13/11/21 5:21:15:183 AEDT] 00000001 ***data***
[13/11/21 5:21:15:276 AEDT] 00000001 ***data***
[13/11/21 5:21:15:278 AEDT] 00000001 ***data***

************ Start Display Current Environment ************

***data***

***data***

***data***

************* End Display Current Environment *************

[17/11/21 5:21:15:183 AEDT] 00000001 ***data***
[17/11/21 5:21:15:276 AEDT] 00000001 ***data***
[17/11/21 5:21:15:278 AEDT] 00000001 ***data***

Please note that the Start and End Display current Environment lines are constant in length and how they start but belong to the timestamp after themselves.

Is there a way to parse this data?

[richgalloway]

Try these props.conf settings:

[mysourcetype]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\*{12} Start
TIME_PREFIX = \[
TIME_FORMAT = %d/%m/%y %H:%M:%S:%$N %Z

[Thomas-R]

Hi @richgalloway,

I tried that but unfortunately it groups the events as per '************ Start' but sometimes the events start with the date instead, it there a way to sometimes get it to parse if this '************ Start' is first unless there is a date there instead?

[richgalloway]

Thanks for clarifying the requirements.  Try this setting.

LINE_BREAKER = ([\r\n]+)(\*{12} Start|\[\d\d\/)