Splunk Search

How do I break multiline events beginning with a #?

ktn01
Path Finder

Hello,

I have to break an event that begins with a # on the first line.

ds-sync-hist: modifyTimestamp:00000167645c911a3acb006bebe0:repl:20181130112407Z
-

# 30/Nov/2018:12:24:09 +0100; conn=-3; op=70042
dn: dc    changetype: modify
replace: ds-sync-state
ds-sync-state: 00000167645c911a3acb006bebe0
ds-sync-state: 000001545149cb662a7e00000008
ds-sync-state: 000001547c812c73545b00000006
ds-sync-state: 0000015451332949677000030def
ds-sync-state: 000001670cc99f4378f6000132aa
-

# 30/Nov/2018:12:26:03 +0100; conn=-2; op=70043
dn: u...    

I tried with BREAK_ONLY_BEFORE = ^\# and BREAK_ONLY_BEFORE = ^#on props.conf but this does not work probably because # is interpreted as the beginning of a comment.

here is the content of my props.conf

[audit]
TIME_PREFIX = ^#\s
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
MAX_TIMESTAMP_LOOKAHEAD = 27
BREAK_ONLY_BEFORE = ^\#
SHOULD_LINEMERGE = true

Thanks

0 Karma

FrankVl
Ultra Champion

Not sure if that break only before would be the right solution anyway, even if the character wouldn't be causing special behavior.

I'd say, try this:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)#

or, to prevent issues with the #

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\x23
TIME_PREFIX = ^\x23\s

https://regex101.com/r/EPG7gB/1

0 Karma

ktn01
Path Finder

Thank's for your answer but my new props.conf does not work better:

[audit]
TIME_PREFIX = ^\x23\s
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
MAX_TIMESTAMP_LOOKAHEAD = 27
BREAK_ONLY_BEFORE = ([\r\n]+)\x23
SHOULD_LINEMERGE = true
0 Karma

FrankVl
Ultra Champion

Ok, so the issue probably wasn't (only) with the # character. As I mentioned: try using line_breaker and linemerge = false, instead of break only before.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

When you run btool props list audit what does it show for TIME_PREFIX and BREAK_ONLY_BEFORE?

---
If this reply helps you, Karma would be appreciated.
0 Karma

ktn01
Path Finder
# /opt//splunkforwarder/bin/splunk btool props list audit                                          
[audit]
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = ([\r\n]+)\x23
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 27
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = true
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
TIME_PREFIX = ^\x23\s
TRANSFORMS =
TRUNCATE = 10000
detect_trailing_nulls = false
maxDist = 500
priority =
sourcetype =
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...