Getting Data In

Logs truncated in Splunk despite line being under the 10000 bytes threshold

victorcorrea
Path Finder

Hi community,

I have observed an issue with the ingestion of the first line in a log file that, at first glance, seemed to have been truncated. Here's a screenshot for reference:

image.png

My apologies for the poor job at blurring the data, but the first event should look like the second event, with a whole lot of data after the highlighted field.

The field DistPoint itself should have a value of "DEPSY.IM2" and, it got, apparently, truncated at such a weird point.

All other subsequent lines in the log were successfully ingested.

There were 3 log files landing on the ingestion point in quick succession - seconds apart, so I am not sure if this could have been the issue.

I was about to update the truncate value for the sourcetype, but all lines in the logs are 3551 bytes, by default.

Any ideas as to what could the problem have been?

Thank you.

0 Karma
1 Solution

victorcorrea
Path Finder

Looks like the issue was with "LINE_MERGE=TRUE" in the props.conf file.

Thank you @PickleRick  and @yuanliu for chiming in.

View solution in original post

0 Karma

victorcorrea
Path Finder

Looks like the issue was with "LINE_MERGE=TRUE" in the props.conf file.

Thank you @PickleRick  and @yuanliu for chiming in.

0 Karma

yuanliu
SplunkTrust
SplunkTrust

Most likely there's some line breaking problem.  Documentation is Configure event line breaking (and the entire Configure event processing.  You would also get better discussion in the forum Getting Data In.

PickleRick
SplunkTrust
SplunkTrust

It might also be the issue with badly/not set EVENT_BREAKER (which is not the same as LINE_BREAKER).

Moving the discussion to Getting Data In.

Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...