Getting Data In

Need help with event linebreaking and timestamp recognition

lksridhar
Explorer

Hi Folks,

Please anyone help me to configure event linebreaking and timestamp recognition for below format logs.

sample logs:


trc file: "dev_w0", trc level: 1, release: "742"

*
* ACTIVE TRACE LEVEL 1
* ACTIVE TRACE COMPONENTS all, MJ
*
M sysno 00
M sid P05
M systemid 390 (AMD/Intel x86_64 with Linux)
M relno 7420
M patchlevel 0
M patchno 439
M intno 20020600
M make multithreaded, Unicode, 64 bit, optimized
M profile /usr/sap/P05/SYS/profile/P05_D00_stp05a02
M pid 3019
M
M

M Sun Sep 17 10:40:23 2017
M kernel runs with dp version 3000(ext=117000) (@(#) DPLIB-INT-VERSION-0+3000-UC)
M ***LOG Q01=> ThInit, WPStart (Workp. 0 1 3019) [thxxhead.c 1052]
M

M Sun Sep 17 10:40:28 2017
M ThInit: running on host stp05a02
I MtxInit: 0 0 0
M calling db_connect ...
B Loading DB library '/usr/sap/P05/D00/exe/dboraslib.so' ...
I

props.conf:

SHOULD_LINEMERGE=false
LINE_BREAKER =
TIME_PREFIX=
MAX_TIMESTAMP_LOOKAHEAD=
TIME_FORMAT=

0 Karma

gjanders
SplunkTrust
SplunkTrust

You can use a TIME_FORMAT of:

TIME_FORMAT = %a %b %d %H:%M:%S %Y

Since the data is a long way into your event you will either have a complicated TIME_PREFIX or a large MAX_TIMESTAMP_LOOKAHEAD.
Is the aim of the line breaker to never break the data ?

LINE_BREAKER = (nolinebreaksplease)

Or does the start of the line with I mark a new line? Which seems strange but ...

LINE_BREAKER = ([\r\n]+)I

I've made a few assumptions here as I'm unsure of exactly what your intentions are.

0 Karma

lksridhar
Explorer

Thanks for the information, my intentions is, i have an log file which don't have time stamp for first 19 lines and "getting the failed to parse timestamp and default to modftime" error on first row.

The below configuration am using for.

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?:\w{1}\s\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4})
TIME_FORMAT = %a %b %d %H:%M:%S %Y

0 Karma

gjanders
SplunkTrust
SplunkTrust

TIME_PREFIX can work over multiple lines as per another answer here, I don't have a very nice solution but this might work:

TIME_PREFIX = ([\r\n]+)(?=M (Sun|Mon|Tue|Wed|Thu|Fri|Sat) [A-Z])

Or perhaps something like this, although I have not tested it (assuming it's always on the next line):

TIME_PREFIX = ([^\n\r]+[\r\n]*){19}
0 Karma

lksridhar
Explorer

I have tried the both and it is not working. still get the same error.
r

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...