topic Re: SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true defaults in Getting Data In

SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true defaults

dglass0215 — Wed, 30 Sep 2020 03:32:53 GMT

Hello,

i am trying to understand the documentation surrounding SHOULD_LINEMERGE. It says the default is SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true. If my understanding is correct this means that if a logfile has multiple lines, the multiple lines will be part of one event that is indexed up until the next date/timestamp is found and then that would be another event.

I have a logfile with the following two lines:

2019-12-30 09:16:41:908: Requestor: IMM_Mobile, IsLocal: False
2019-12-30 09:16:41:908: 637132942019089151: Scanned CID: BARCODE:

However, they get indexed as one event and this is not what I want. A previous suggestion to me was to make SHOULD_LINEMERGE = false, however I do not think I want to do that because there are certain entries in the logfile that do span multiple lines.

Any assistance is greatly appreciated!
David

Re: SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true defaults

richgalloway — Wed, 08 Jan 2020 18:48:28 GMT

Your understanding of those two settings is correct. It's important, however, to define what constitutes a date. Add TIME_FORMAT = %Y-%m-%d %H:%M:%S:%3N to the appropriate props.conf file.

Re: SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true defaults

dglass0215 — Thu, 09 Jan 2020 14:59:54 GMT

Thanks! So far so good. I have not seen an incorrect merging of lines since I added TIME_FORMAT. Will keep monitoring and will mark this as the answer soon.