Getting Data In

Need help with event linebreaking and timestamp recognition

lksridhar
Explorer

Hi Folks,

Please anyone help me to configure event linebreaking and timestamp recognition for below format logs.

sample logs:


trc file: "dev_w0", trc level: 1, release: "742"

*
* ACTIVE TRACE LEVEL 1
* ACTIVE TRACE COMPONENTS all, MJ
*
M sysno 00
M sid P05
M systemid 390 (AMD/Intel x86_64 with Linux)
M relno 7420
M patchlevel 0
M patchno 439
M intno 20020600
M make multithreaded, Unicode, 64 bit, optimized
M profile /usr/sap/P05/SYS/profile/P05_D00_stp05a02
M pid 3019
M
M

M Sun Sep 17 10:40:23 2017
M kernel runs with dp version 3000(ext=117000) (@(#) DPLIB-INT-VERSION-0+3000-UC)
M ***LOG Q01=> ThInit, WPStart (Workp. 0 1 3019) [thxxhead.c 1052]
M

M Sun Sep 17 10:40:28 2017
M ThInit: running on host stp05a02
I MtxInit: 0 0 0
M calling db_connect ...
B Loading DB library '/usr/sap/P05/D00/exe/dboraslib.so' ...
I

props.conf:

SHOULD_LINEMERGE=false
LINE_BREAKER =
TIME_PREFIX=
MAX_TIMESTAMP_LOOKAHEAD=
TIME_FORMAT=

0 Karma

gjanders
SplunkTrust
SplunkTrust

You can use a TIME_FORMAT of:

TIME_FORMAT = %a %b %d %H:%M:%S %Y

Since the data is a long way into your event you will either have a complicated TIME_PREFIX or a large MAX_TIMESTAMP_LOOKAHEAD.
Is the aim of the line breaker to never break the data ?

LINE_BREAKER = (nolinebreaksplease)

Or does the start of the line with I mark a new line? Which seems strange but ...

LINE_BREAKER = ([\r\n]+)I

I've made a few assumptions here as I'm unsure of exactly what your intentions are.

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma

lksridhar
Explorer

Thanks for the information, my intentions is, i have an log file which don't have time stamp for first 19 lines and "getting the failed to parse timestamp and default to modftime" error on first row.

The below configuration am using for.

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?:\w{1}\s\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4})
TIME_FORMAT = %a %b %d %H:%M:%S %Y

0 Karma

gjanders
SplunkTrust
SplunkTrust

TIME_PREFIX can work over multiple lines as per another answer here, I don't have a very nice solution but this might work:

TIME_PREFIX = ([\r\n]+)(?=M (Sun|Mon|Tue|Wed|Thu|Fri|Sat) [A-Z])

Or perhaps something like this, although I have not tested it (assuming it's always on the next line):

TIME_PREFIX = ([^\n\r]+[\r\n]*){19}
Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma

lksridhar
Explorer

I have tried the both and it is not working. still get the same error.
r

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!