topic Re: Need help with event linebreaking and timestamp recognition in Getting Data In

Need help with event linebreaking and timestamp recognition

lksridhar — Tue, 29 Sep 2020 16:07:30 GMT

Hi Folks,

Please anyone help me to configure event linebreaking and timestamp recognition for below format logs.

sample logs:

trc file: "dev_w0", trc level: 1, release: "742"

*
* ACTIVE TRACE LEVEL 1
* ACTIVE TRACE COMPONENTS all, MJ
*
M sysno 00
M sid P05
M systemid 390 (AMD/Intel x86_64 with Linux)
M relno 7420
M patchlevel 0
M patchno 439
M intno 20020600
M make multithreaded, Unicode, 64 bit, optimized
M profile /usr/sap/P05/SYS/profile/P05_D00_stp05a02
M pid 3019
M
M

M Sun Sep 17 10:40:23 2017
M kernel runs with dp version 3000(ext=117000) (@(#) DPLIB-INT-VERSION-0+3000-UC)
M ***LOG Q01=> ThInit, WPStart (Workp. 0 1 3019) [thxxhead.c 1052]
M

M Sun Sep 17 10:40:28 2017
M ThInit: running on host stp05a02
I MtxInit: 0 0 0
M calling db_connect ...
B Loading DB library '/usr/sap/P05/D00/exe/dboraslib.so' ...
I

props.conf:

SHOULD_LINEMERGE=false
LINE_BREAKER =
TIME_PREFIX=
MAX_TIMESTAMP_LOOKAHEAD=
TIME_FORMAT=

Re: Need help with event linebreaking and timestamp recognition

gjanders — Tue, 29 Sep 2020 16:12:29 GMT

You can use a TIME_FORMAT of:

TIME_FORMAT = %a %b %d %H:%M:%S %Y

Since the data is a long way into your event you will either have a complicated TIME_PREFIX or a large MAX_TIMESTAMP_LOOKAHEAD.
Is the aim of the line breaker to never break the data ?

LINE_BREAKER = (nolinebreaksplease)

Or does the start of the line with I mark a new line? Which seems strange but ...

LINE_BREAKER = ([\r\n]+)I

I've made a few assumptions here as I'm unsure of exactly what your intentions are.

Re: Need help with event linebreaking and timestamp recognition

lksridhar — Tue, 29 Sep 2020 16:07:42 GMT

Thanks for the information, my intentions is, i have an log file which don't have time stamp for first 19 lines and "getting the failed to parse timestamp and default to modftime" error on first row.

The below configuration am using for.

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?:\w{1}\s\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4})
TIME_FORMAT = %a %b %d %H:%M:%S %Y

Re: Need help with event linebreaking and timestamp recognition

gjanders — Mon, 09 Oct 2017 12:29:43 GMT

TIME_PREFIX can work over multiple lines as per another answer here, I don't have a very nice solution but this might work:

TIME_PREFIX = ([\r\n]+)(?=M (Sun|Mon|Tue|Wed|Thu|Fri|Sat) [A-Z])

Or perhaps something like this, although I have not tested it (assuming it's always on the next line):

TIME_PREFIX = ([^\n\r]+[\r\n]*){19}

Re: Need help with event linebreaking and timestamp recognition

lksridhar — Mon, 09 Oct 2017 14:25:06 GMT

I have tried the both and it is not working. still get the same error.
r