Getting Data In

How to remove an invalid line breaker from syslog before indexing?

Explorer

Hi everyone,

I've got an application sending data to splunk, which are split over multiple lines instead to keep everything on the same line.

When I redirect my data to a file instead of splunk, I can find that the ascii code #012 is sent as part of the string.

Example:

... #012Change details       : #012filewrite#012 ...

Which are split in multiples lines in splunk:

...
9/29/16 3:25:30.000 AM  filewrite
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
9/29/16 3:25:30.000 AM  Change details       :
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
...

Is there any way to replace the ASCII code #012 before to index it into splunk ?

I've try to add this config in my props.conf, but it did not solved it.

[syslog]
LINE_BREAKER=#012
SHOULD_LINEMERGE=true

And also this one:

[syslog]
SEDCMD-fim = s/\#012/ /g

Thanks for your support.

0 Karma
1 Solution

Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINEBREAKER=([\r\n]+)
SHOULD
LINEMERGE=true

My data are now merged correctly in one line.

View solution in original post

0 Karma

Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINEBREAKER=([\r\n]+)
SHOULD
LINEMERGE=true

My data are now merged correctly in one line.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

The #012 is ascii code for \n, so that is why events are getting split (default LINE_BREAKER is ([\r\n]+)). You second configuration with SEDCMD will not work as SEDCMD executes after events are broken.

How does your data looks like when you redirect it to a file (sample entries, mask any sensitive data)? You probably have to setup correct LINE_BREAKER (assuming you're data is directly coming to indexer/heavy forwarder) to split it correctly.

Explorer

Hi somesoni2,

Actually, I did not test it with the default LINEBREAKER, as I was fighting with my #012 value.
Setting the default value LINE
BREAKER and the SHOULD_LINEMERGE is working fine.

So, I've put the following in my props.conf:
[syslog]
LINEBREAKER=([\r\n]+)
SHOULD
LINEMERGE=true

Thanks a lot.
Have a great day.

0 Karma