Archive
Highlighted

How to event break on multiple dashes?

New Member

With multi-line logs, I am trying to linebreak on an obvious linebreaker of dashes (----------------------------------------------------------). (Note in the below examples it appears to be coming across as a whole line, but it should be like above).

Example log:

ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
EventId : 300
Keywords : 4
Level : Informational
Message : Application information
Opcode : Info
Task : 65234
Version : 0
Payload : Generic information
EventName : InfoInfo
ProcessId : 6528
ThreadId : 12524

Timestamp : 2019-08-30 12:32:50 PM

I've tried various regex expressions, one such as ^(\s+)-+(\s+)$ to break on the line, but the results don't seem to work. Also Splunk seems to interpret the Timestamp as the beginning of the log but it is actually the last part of the log before the linebreak.

In general Splunk will display the events as (note the Timestamp is first, but it should be last):

Example results:

Timestamp : 2019-08-30 12:32:50 PM

ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
EventId : 300
Keywords : 4
...

0 Karma
Highlighted

Re: How to event break on multiple dashes?

SplunkTrust
SplunkTrust

try setting BREAK_ONLY_BEFORE = ProviderId in your props.conf

0 Karma
Highlighted

Re: How to event break on multiple dashes?

New Member

That seems like a good idea actually, but for some reason it is tending to bunch up a lot of events together, or still putting the timestamp at the top. In some cases it does take out the Timestamp and includes it only as part of the Event Time itself (which is fine). I modified the props.conf within \etc\system\local and restarted the service.

The props.conf config looks like this:

[sourceTypeName]
BREAKONLYBEFORE = ProviderId

Splunk Event examples:

... 1 line omitted ...
ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
... 13 lines omitted ...
ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
... 13 lines omitted ...
ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
... 13 lines omitted ...
ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84
... 13 lines omitted ...
ProviderId : 453af5ee-6772-55ce-39b3-0f9307a96b84

0 Karma
Highlighted

Re: How to event break on multiple dashes?

SplunkTrust
SplunkTrust

you have only BREAKONLYBEFORE in props.conf for that stanza?

could you share the entire configuration for that sourcetype?

0 Karma
Highlighted

Re: How to event break on multiple dashes?

New Member

Sure, here (from the Splunk UI). They are basically the default settings I believe:

Name Value

CHARSET AUTO
DATETIMECONFIG [blank]
LINE
BREAKER ([\r\n]+)
NOBINARYCHECK true
SHOULDLINEMERGE true
category Custom
disabled false
pulldown
type true

0 Karma
Highlighted

Re: How to event break on multiple dashes?

New Member

And yes, that is the only line for the stanza. Created the props.conf file specifically for it.

0 Karma
Highlighted

Re: How to event break on multiple dashes?

SplunkTrust
SplunkTrust

why there is LINE_BREAKER? when you have breakonlybefore
comment all that and try this new :

    [your_sourcetype]
    CHARSET =
    SHOULD_LINEMERGE = true
    NO_BINARY_CHECK = true
    BREAK_ONLY_BEFORE = ProviderId
    TIME_FORMAT = %Y-%m-%d %I:%M:%S %p
    TIME_PREFIX = Timestamp\s:\s

and change it from the backend

0 Karma
Highlighted

Re: How to event break on multiple dashes?

New Member

The above was taken directly from the Splunk UI, which autogenerated that LINE_BREAKER.

In props.conf, it is as you've just described.

Could there be any conflicts?

0 Karma
Highlighted

Re: How to event break on multiple dashes?

New Member

One more point to note, after the dashed line is a blank line. The line could break on the blank line instead of the dashes.

0 Karma
Highlighted

Re: How to event break on multiple dashes?

Path Finder

Try adding MAXTIMESTAMPLOOKAHEAD to your props stanza:

MAXTIMESTAMPLOOKAHEAD = 320

if the content length varies, use a value appropriate for the variance:

MAXTIMESTAMPLOOKAHEAD = 512

I use multiples of 64 on x86-64 "just in case" Splunk allocates this as a separate buffer. Different architectures have different cache line sizes.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.