topic Re: Event parsing issues in Installation

Event parsing issues

sathishkompelly — Wed, 30 Sep 2020 01:12:36 GMT

Hi,
We are seeing very often event parsing issues when monitoring a file, we added the below settings but still it is not breaking at the right point, it is breaking at uneven points like below samples. We are trying to parse the winevent logs. Please help me to get this parsed at the date point.

props.conf:-
[host::host01]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = true
TRUNCATE = 10000

Sample Logs:-

7/10/19

10:27:22.000 AM
que identifier that can be used to xxxxxx-some text -xxxxx-xxxxxx - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

7/10/19
9:40:34.000 AM

identifier that can be xxxxxx-some text -xxxxx-xxxxxxx.

7/10/19
9:31:21.000 AM

ogged on. xxxx some text xxxxxxx This will be 0 if no session key was requested.

Re: Event parsing issues

ragedsparrow — Wed, 10 Jul 2019 18:35:29 GMT

Greetings,

So, in a props.conf, there are 6 base settings that should be defined (also known as the "magic six"). They are as follows:

SHOULD_LINEMERGE=
LINE_BREAKER=
MAX_TIMESTAMP_LOOKAHEAD=
TIME_PREFIX=
TRUNCATE=
TIME_FORMAT=

Here is a good resource: https://www.hurricanelabs.com/splunk-tutorials/splunk-baseline-props-part-1-building-a-smart-foundation

For what you've posted as your sample data, I would do something like this:

[host::host01]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)(?=\d+\/)
NO_BINARY_CHECK=true
TIME_FORMAT=%D%n%T.%3N%n%p
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=26
TRUNCATE=1000

Re: Event parsing issues

sathishkompelly — Wed, 30 Sep 2020 01:12:44 GMT

@ragedsparrow Thanks for the reply, when I added the settings it is taking multiple lines to a single event,

7/10/19
2:35:11.000 PM

Jul 10 12:34:56 xxxxxx-some text -xxxxx-xxxxxx - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Jul 10 12:35:00 xxxxxx-some text -xxxxx-xxxxxxx.

SHOULD_LINEMERGE = true
LINE_BREAKER = ([\r\n]+)(?=\d+\/)
NO_BINARY_CHECK = true
TIME_FORMAT = %D%n%T.%3N%n%p
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 7
TRUNCATE = 1000

Re: Event parsing issues

ragedsparrow — Wed, 10 Jul 2019 19:58:42 GMT

@sathishkompelly , in your original post you had different timestamps:

    7/10/19

10:27:22.000 AM
que identifier that can be used to xxxxxx-some text -xxxxx-xxxxxx - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

7/10/19
9:40:34.000 AM
identifier that can be xxxxxx-some text -xxxxx-xxxxxxx.

7/10/19
9:31:21.000 AM
ogged on. xxxx some text xxxxxxx This will be 0 if no session key was requested.

In this reply, you are giving a different timestamp:

7/10/19
2:35:11.000 PM
Jul 10 12:34:56 xxxxxx-some text -xxxxx-xxxxxx - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Jul 10 12:35:00 xxxxxx-some text -xxxxx-xxxxxxx.

If the timestamp format changes, then this won't work. If it remains the same, as you originally posted, it should work