Getting Data In

How to remove an invalid line breaker from syslog before indexing?

vlours
Explorer

Hi everyone,

I've got an application sending data to splunk, which are split over multiple lines instead to keep everything on the same line.

When I redirect my data to a file instead of splunk, I can find that the ascii code #012 is sent as part of the string.

Example:

... #012Change details       : #012filewrite#012 ...

Which are split in multiples lines in splunk:

...
9/29/16 3:25:30.000 AM  filewrite
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
9/29/16 3:25:30.000 AM  Change details       :
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
...

Is there any way to replace the ASCII code #012 before to index it into splunk ?

I've try to add this config in my props.conf, but it did not solved it.

[syslog]
LINE_BREAKER=#012
SHOULD_LINEMERGE=true

And also this one:

[syslog]
SEDCMD-fim = s/\#012/ /g

Thanks for your support.

0 Karma
1 Solution

vlours
Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

My data are now merged correctly in one line.

View solution in original post

0 Karma

vlours
Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

My data are now merged correctly in one line.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The #012 is ascii code for \n, so that is why events are getting split (default LINE_BREAKER is ([\r\n]+)). You second configuration with SEDCMD will not work as SEDCMD executes after events are broken.

How does your data looks like when you redirect it to a file (sample entries, mask any sensitive data)? You probably have to setup correct LINE_BREAKER (assuming you're data is directly coming to indexer/heavy forwarder) to split it correctly.

vlours
Explorer

Hi somesoni2,

Actually, I did not test it with the default LINE_BREAKER, as I was fighting with my #012 value.
Setting the default value LINE_BREAKER and the SHOULD_LINEMERGE is working fine.

So, I've put the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

Thanks a lot.
Have a great day.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...