Getting Data In

events are broken in the middle of the line

mataharry
Communicator

I am monitoring with a forwarder logs file that are being written.
And sometimes the events indexed are broken in multiple events in the middle of the line :

example :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and the end of my line
becomes
event1 :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and
event2
the end of my line

1 Solution

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

mataharry
Communicator

I confirm the application generating the log file is writing by chunks.
It doesn't happen if I index the file after.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...