topic Re: How do I break multiline events beginning with a #? in Splunk Search

How do I break multiline events beginning with a #?

ktn01 — Fri, 30 Nov 2018 13:08:27 GMT

Hello,

I have to break an event that begins with a # on the first line.

ds-sync-hist: modifyTimestamp:00000167645c911a3acb006bebe0:repl:20181130112407Z
-

# 30/Nov/2018:12:24:09 +0100; conn=-3; op=70042
dn: dc    changetype: modify
replace: ds-sync-state
ds-sync-state: 00000167645c911a3acb006bebe0
ds-sync-state: 000001545149cb662a7e00000008
ds-sync-state: 000001547c812c73545b00000006
ds-sync-state: 0000015451332949677000030def
ds-sync-state: 000001670cc99f4378f6000132aa
-

# 30/Nov/2018:12:26:03 +0100; conn=-2; op=70043
dn: u...

I tried with BREAK_ONLY_BEFORE = ^\# and BREAK_ONLY_BEFORE = ^#on props.conf but this does not work probably because # is interpreted as the beginning of a comment.

here is the content of my props.conf

[audit]
TIME_PREFIX = ^#\s
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
MAX_TIMESTAMP_LOOKAHEAD = 27
BREAK_ONLY_BEFORE = ^\#
SHOULD_LINEMERGE = true

Thanks

Re: How do I break multiline events beginning with a #?

richgalloway — Tue, 29 Sep 2020 22:09:41 GMT

When you run btool props list audit what does it show for TIME_PREFIX and BREAK_ONLY_BEFORE?

Re: How do I break multiline events beginning with a #?

FrankVl — Fri, 30 Nov 2018 13:34:58 GMT

Not sure if that break only before would be the right solution anyway, even if the character wouldn't be causing special behavior.

I'd say, try this:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)#

or, to prevent issues with the #

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\x23
TIME_PREFIX = ^\x23\s

https://regex101.com/r/EPG7gB/1

Re: How do I break multiline events beginning with a #?

ktn01 — Fri, 30 Nov 2018 14:11:06 GMT

Thank's for your answer but my new props.conf does not work better:

[audit]
TIME_PREFIX = ^\x23\s
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
MAX_TIMESTAMP_LOOKAHEAD = 27
BREAK_ONLY_BEFORE = ([\r\n]+)\x23
SHOULD_LINEMERGE = true

Re: How do I break multiline events beginning with a #?

ktn01 — Fri, 30 Nov 2018 14:41:43 GMT

# /opt//splunkforwarder/bin/splunk btool props list audit                                          
[audit]
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = ([\r\n]+)\x23
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 27
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = true
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
TIME_PREFIX = ^\x23\s
TRANSFORMS =
TRUNCATE = 10000
detect_trailing_nulls = false
maxDist = 500
priority =
sourcetype =

Re: How do I break multiline events beginning with a #?

FrankVl — Fri, 30 Nov 2018 15:47:58 GMT

Ok, so the issue probably wasn't (only) with the # character. As I mentioned: try using line_breaker and linemerge = false, instead of break only before.