Getting Data In

Need help with event linebreaking and timestamp recognition

lksridhar
Explorer

Hi Folks,

Please anyone help me to configure event linebreaking and timestamp recognition for below format logs.

sample logs:


trc file: "dev_w0", trc level: 1, release: "742"

*
* ACTIVE TRACE LEVEL 1
* ACTIVE TRACE COMPONENTS all, MJ
*
M sysno 00
M sid P05
M systemid 390 (AMD/Intel x86_64 with Linux)
M relno 7420
M patchlevel 0
M patchno 439
M intno 20020600
M make multithreaded, Unicode, 64 bit, optimized
M profile /usr/sap/P05/SYS/profile/P05_D00_stp05a02
M pid 3019
M
M

M Sun Sep 17 10:40:23 2017
M kernel runs with dp version 3000(ext=117000) (@(#) DPLIB-INT-VERSION-0+3000-UC)
M ***LOG Q01=> ThInit, WPStart (Workp. 0 1 3019) [thxxhead.c 1052]
M

M Sun Sep 17 10:40:28 2017
M ThInit: running on host stp05a02
I MtxInit: 0 0 0
M calling db_connect ...
B Loading DB library '/usr/sap/P05/D00/exe/dboraslib.so' ...
I

props.conf:

SHOULD_LINEMERGE=false
LINE_BREAKER =
TIME_PREFIX=
MAX_TIMESTAMP_LOOKAHEAD=
TIME_FORMAT=

0 Karma

gjanders
SplunkTrust
SplunkTrust

You can use a TIME_FORMAT of:

TIME_FORMAT = %a %b %d %H:%M:%S %Y

Since the data is a long way into your event you will either have a complicated TIME_PREFIX or a large MAX_TIMESTAMP_LOOKAHEAD.
Is the aim of the line breaker to never break the data ?

LINE_BREAKER = (nolinebreaksplease)

Or does the start of the line with I mark a new line? Which seems strange but ...

LINE_BREAKER = ([\r\n]+)I

I've made a few assumptions here as I'm unsure of exactly what your intentions are.

0 Karma

lksridhar
Explorer

Thanks for the information, my intentions is, i have an log file which don't have time stamp for first 19 lines and "getting the failed to parse timestamp and default to modftime" error on first row.

The below configuration am using for.

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?:\w{1}\s\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4})
TIME_FORMAT = %a %b %d %H:%M:%S %Y

0 Karma

gjanders
SplunkTrust
SplunkTrust

TIME_PREFIX can work over multiple lines as per another answer here, I don't have a very nice solution but this might work:

TIME_PREFIX = ([\r\n]+)(?=M (Sun|Mon|Tue|Wed|Thu|Fri|Sat) [A-Z])

Or perhaps something like this, although I have not tested it (assuming it's always on the next line):

TIME_PREFIX = ([^\n\r]+[\r\n]*){19}
0 Karma

lksridhar
Explorer

I have tried the both and it is not working. still get the same error.
r

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...