Archive
Highlighted

Can you help me with Line Breaker and Event Time?

Contributor

Hello,

I have a database crashdump file, which has the following structure (from the beginning):

=========================================
Crash protocol for PID 289251
Rundirectory: /usr/sap/CWW/HDB02/ls5433
Process start time: 2018-11-29 15:24:13 782 Local
Exception time: 2018-11-30 21:00:50 859 Local
SectionTimeout: 30sec
KillTimeout: 300sec
=========================================

Table of contents:
   [BUILD]  Build information
   [SYSTEMINFO]  System information
   [CRASH_SHORTINFO]  Exception short info
   ...
   [DISASSEMBLY]  Disassembly of frames in callstack
   [REGISTER_OBJECTS]  Objects registers point to
   [THREADS]  Running threads
WARNING: could not suspend 1 threads

[BUILD]  Build information: (2018-11-30 21:01:01 972 Local)
Version       : 2.00.024.06.1538035880 (fa/hana2sp02)
Build host    : ld4552
Build time    : 2018-09-27 10:26:47
Platform      : linuxx86_64
Compiler      : gcc (SAP release 20170307, based on SUSE gcc6-6.2.1+r239768-2.4) 6.2.1 20160826 [gcc-6-branch revision 239773]
Maketype      : rel
Branch        : fa/hana2sp02
Git hash      : 4d5d5af986c75c6f2d61bde8b289b9c8ca078032
Git mergetime : 2018-09-27 10:11:20
Weekstone     : 0000.00.0
[OK]
--

[SYSTEMINFO]  System information: (2018-11-30 21:01:01 972 Local)
Instance CWW/02, OS Linux ls5433 4.4.121-92.98-default #1 SMP Fri Oct 19 07:52:13 UTC 2018 (e4d85ce) x86_64
[OK]
--

Now, I would like each section to be a separate event. The sections are separated with the [OK], except the heading one, but here I would not have anything against that it is combined together with the [BUILD] section together. This means I would like to set the line breaker to the [OK].

Also, the event time should be set to the timestamp coming in brackets:

[BUILD]  Build information: (2018-11-30 21:01:01 972 Local)
...
[SYSTEMINFO]  System information: (2018-11-30 21:01:01 972 Local)

etc. Unfortunately these timestamps do not come in the first line of the event, at least in case of the first event / header.

How would I achieve both?

The line breaker set to [OK] and the proper event time setting?

Could you please advice the corresponding props.conf entries?

Kind Regards,
Kamil

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

Legend

Hi damucka,
let me understand: you want to break your file using [OK] as Line Breaker and you want as timestamp the date and time that is between round brackets, is it correct?

If this is your requirements, try this props.conf:

[ my_sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\[OK\]
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=\(

Bye.
Giuseppe

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

SplunkTrust
SplunkTrust

Hi,

Please try below configuration

props.conf

[yoursourcetype]
BREAK_ONLY_BEFORE_DATE = false
LINE_BREAKER = ([\r\n+])\[\w+\]\s{2}\w+
MAX_TIMESTAMP_LOOKAHEAD = 23
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S %3N
TIME_PREFIX = information: \(
disabled = false

Above configuration will break sample event provided by you in 3 different events
First Event

=========================================
Crash protocol for PID 289251
Rundirectory: /usr/sap/CWW/HDB02/ls5433
Process start time: 2018-11-29 15:24:13 782 Local
Exception time: 2018-11-30 21:00:50 859 Local
SectionTimeout: 30sec
KillTimeout: 300sec
=========================================

Table of contents:
   [BUILD]  Build information
   [SYSTEMINFO]  System information
   [CRASH_SHORTINFO]  Exception short info
   ...
   [DISASSEMBLY]  Disassembly of frames in callstack
   [REGISTER_OBJECTS]  Objects registers point to
   [THREADS]  Running threads
WARNING: could not suspend 1 threads

Second Event

[BUILD]  Build information: (2018-11-30 21:01:01 972 Local)
Version       : 2.00.024.06.1538035880 (fa/hana2sp02)
Build host    : ld4552
Build time    : 2018-09-27 10:26:47
Platform      : linuxx86_64
Compiler      : gcc (SAP release 20170307, based on SUSE gcc6-6.2.1+r239768-2.4) 6.2.1 20160826 [gcc-6-branch revision 239773]
Maketype      : rel
Branch        : fa/hana2sp02
Git hash      : 4d5d5af986c75c6f2d61bde8b289b9c8ca078032
Git mergetime : 2018-09-27 10:11:20
Weekstone     : 0000.00.0
[OK]
--

Third Event

[SYSTEMINFO]  System information: (2018-11-30 21:01:01 972 Local)
Instance CWW/02, OS Linux ls5433 4.4.121-92.98-default #1 SMP Fri Oct 19 07:52:13 UTC 2018 (e4d85ce) x86_64
[OK]
--

View solution in original post

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

Contributor

Hello harsmarvania57,

Thank you.
Could you perhaps say how the first event will get the event time set in this case?
The string "information: (" is not there.
Also, the rest of the log entries look like the 3-rd one, separated with the [OK]. Will they be translated properly split and given the timestamp like in the case of the 3-rd one?

Kind Regards,
Kamil

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

SplunkTrust
SplunkTrust

Hi Kamil,

For the First event, splunk will not able to find timestamp because I have explicitly given TIME_PREFIX = information: \( so in this case splunk will assign system time when event actually parsed by splunk.

If you want to set timestamp for the First Event then you can change TIME_PREFIX as given below.

If you want Process start time: then TIME_PREFIX = (information: \(|Process start time: )

If you want Exception time: then TIME_PREFIX = (information: \(|Exception time: )

I didn't get what you are trying to say for 3rd one but splunk will assign timestamp as 2018-11-30 21:01:01 972 to 3rd event. Is this what you are looking for?

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

Contributor

Hello,

Thank you, I think the
TIME_PREFIX = (information: (|Exception time: )

is the option to choose. With the 3rd event I meant, that the file is big and has many sections/logs/events which look precisely like the 3rd log. I wanted to just reassure that the configuration above will correctly get them, but I guess this became obvious to me now.

Thank you, I will check the configuration proposed by you.

Kind Regards,
Kamil

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

SplunkTrust
SplunkTrust

Best to test this configuration in Standalone or Test environment and then implement it in production. I have converted my comment to answer and if it will work then you can accept it.

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

Contributor

Hello,

One more question.
Unfortunately I noticed first now that the date prefix is not always "information: (", so sometimes the event time is set wrong. For example:
[CRASH_EMERGENCYSTACK] Emergency stacktrace: (2018-11-30 21:01:01 973 Local)

Also when I would like to define the prefix as only ": (" it will not work as there are other strings with this suffix. In principle I would need to define the prefix using the regex somehow, telling that this would be the line with the [] brackets and having the ": (" as a prefix.
Is it possible?

Kind regards,
Kamil

0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

SplunkTrust
SplunkTrust

Hi Kamil,

Please try below config in props.conf

[yoursourcetype]
BREAK_ONLY_BEFORE_DATE=false
CHARSET=UTF-8
LINE_BREAKER=([\r\n+])\[\w+\]\s+\w+
MAX_TIMESTAMP_LOOKAHEAD=23
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S %3N
TIME_PREFIX=(?:\[\w+\]\s+\w+(?:\s+\w+)?\:\s\(|Exception\stime\:\s)
disabled=false
0 Karma
Highlighted

Re: Can you help me with Line Breaker and Event Time?

Contributor

Hello harsmanvania57,

Thank you.
The above configuration works for all sections of the crashdump file but one and I do not why.
For the section below the timestamps seems not to be recognized correctly and the event time is set wrongly:

[CRASH_EXTINFO]  Extended exception info: (2018-12-03 12:29:34 058 Local)
----> Dump of siginfo contents <----
  signal:      11(SIGSEGV)
  code:        STACK OVERFLOW: 2(SEGV_ACCERR: invalid permissions for mapped object)

The event time set is 12:29:34.015 PM, so it is taken from the previous section.
The next section after that gets recognized already correctly (timestamp) and the event time is set in a right way:

[CRASH_CONTEXT]  Context info: (2018-12-03 12:29:34 058 Local)
----> Crashing context information <----
  ContextStack at (0x00007f3b4e31c460)

The event time set in this case is 12:29:34.058 PM, so it is correct.

Any hint why in this one case the timestamp does not get recognized correctly?

Kind Regards,
Kamil

0 Karma