Deployment Architecture

Splunk timing issue when forwarding log file

brent_weaver
Builder

I am having an issue with a particular log file where two entries get concatenated into one entry. It is not the data because if I take take the same data and add it via file upload it is fine, meaning all lines are unique. Has anyone else had an issue like this? It seems that it is a timing problem that the host if forwarding the same time the file is being written... Is this perhaps the result of the log file not being locked when written to???

Any help is appreciated. Thanks!

0 Karma

pradeepkumarg
Influencer

You need to set line breaking property in your props.conf

http://docs.splunk.com/Documentation/Splunk/6.2.5/Admin/propsconf

LINE_BREAKER = <regular expression>
* Specifies a regex that determines how the raw text stream is broken into initial events,
0 Karma

brent_weaver
Builder

Does this still apply when the issue only happen sometimes? It is the same file, it just seems that the file get fwd before it is done being written.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Reuse: Word-play

This puzzle series (first published here) is based on analysing words.Some of the statements in your puzzle ...

Tuning Edge Processor in Hybrid Mode: What Limits Each Data Path

When you run Edge Processor in hybrid mode, alongside a Splunk heavy forwarder on the same host, two questions ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...