Getting Data In
Highlighted

How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Builder

I have syslog messages arriving at the indexer with embedded ASCII form feed characters (#012). Splunk is breaking on these characters, and I want to avoid this. How can I tell Splunk not to break on these characters?

Here is part of a message for illustration - Splunk is breaking this message into three events, one at each "#012":

Calling-Station-ID#011 value:a8-40-41-14-df-5c#012#011Attribute:CreateTime#011 value:1471359492123#012#011Attribute:DestinationIPAddress#011 value:10.51.17.211#012#011Attribute:Device Identifier#011
0 Karma
Highlighted

Re: How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Contributor

You could make your own sourcetype in the props.conf

there you can use the parameter
LINE_BREAKER = [regular expression]

like

[yourSourcetype]
LINE_BREAKER=\r\nCalling-Station.*
0 Karma
Highlighted

Re: How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Builder

Thanks for the reply. As I mentioned, the message I noted above is only part of a much larger message. The beginning of the message has a timestamp, which Splunk seems to be identifying correctly. But I have BREAKONLYBEFORE_DATE = true, so I don't understand why Splunk would also break at these #012 characters. Clearly the part of the message before those characters is not a timestamp.

I am pretty green, so I opened a support case asking for some assistance and education, but support has been very unhelpful.

What makes this so complex to me is that these messages are also processed by the Cisco ISE TA. My input applies a sourcetype=syslog, but when I view the messages, they have a sourcetype=cisco:ise:syslog. Being inexperienced, I am not quite clear on how this transition happens. But I can tell you that both my syslog sourcetype and cisco:ise:syslog sourcetypes both have BREAKONLYBEFORE_DATE = true.

0 Karma
Highlighted

Re: How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Influencer

I was only using Calling-Station-ID as an example. Using LINE_BREAKER is preferred, in my experience, over BOBD.

0 Karma
Highlighted

Re: How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Contributor

You can do it like this:

 [yourSourcetype]
 LINE_BREAKER=(\r\n+)\d{2}-\d{2}-\d{4}

If your event starts like
20-06-2016 .....
as twinspop said. LINEBREAKER is preferred over BREAK:ONLYBEFORE_DATE

0 Karma
Highlighted

Re: How to configure Splunk to prevent line breaking events on ASCII character "#012"?

Influencer

In props.conf on the indexer(s) explicitly define what the line break should be:

[yoursourcetype]
LINE_BREAKER = ([\r\n]+)Calling-Station-ID
SHOULD_LINEMERGE = false

Line breaker needs 1 capturing group. Anything matched in the group will not be indexed. In the case above, newline or carriage return followed by Calling-Station-ID marks the beginning of a new event. The newline/CR is discarded and not indexed.