Getting Data In

events are broken in the middle of the line

mataharry
Communicator

I am monitoring with a forwarder logs file that are being written.
And sometimes the events indexed are broken in multiple events in the middle of the line :

example :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and the end of my line
becomes
event1 :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and
event2
the end of my line

1 Solution

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

mataharry
Communicator

I confirm the application generating the log file is writing by chunks.
It doesn't happen if I index the file after.

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...