Knowledge Management

To rename-and-recreate or to copy-truncate in log rotate - so that most real time data is got and no data is lost even in edge cases?

uljasmi1veikkau
Engager

Hi,

After reading:
- https://answers.splunk.com/answers/49663/log-rotation-best-practices.html
- https://answers.splunk.com/answers/436074/why-does-data-stop-getting-indexed-after-a-log-rot.html

I'm bit puzzled what is the best practice way:
- to get most real time data in
- not to lose any data during rotations

And how should Splunk be set to monitor rotated paths of the files (e.g. set to monitor also compressed paths of the rotated files or to exclude them from the monitors). So that also most part of the edge cases would have been covered:
- e.g. maybe splunk forwarder is not running in some point in time
- e.g. maybe splunk forwarder is not able to send data to Indexers at point in time
- e.g. maybe remnants of the data is written by application to the rotated file.

If I want both. To get the most real time data in and not to lose any data.

Should I rename-and-recreate or to copy-truncate?
Should I set Splunk to also monitor rotated and compressed paths? Even if there is "months" of compressed log files in path..

Tags (2)
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...