Splunk Enterprise

Log rotation affecting reading of logs

sphiwee
Contributor

Hi there we noticed we are not getting some logs coming through @ some hours in the morning after log rotation. so we ran the below query.

 

index=_internal host=* /opt/workfusion/supervisord/log/workfusion.out.log NOT Metrics earliest=-7d latest=now
| timechart span=5m count as NumInt

 

here's the result below

11-24-2020 01:30:03.080 +0200 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/workfusion/supervisord/log/workfusion.out.log'.

 

11-19-2020 01:30:04.536 +0200 INFO WatchedFile - Logfile truncated while open, original pathname file='/opt/workfusion/supervisord/log/workfusion.out.log', will begin reading from start.

 

How can I fix this since it's affecting our dashboard because there are no results or logs so the dashboard is empty. 

 

 

 

Labels (3)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
How you are rotating those logs? mv + touch, cp + truncate something else?
0 Karma

sphiwee
Contributor

cp to backup location, then rm originals 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Maybe you should try logrotate to rotate log files e.g. https://linux.die.net/man/8/logrotate
You could try different options how to signal your software to release filehandlers to old log file and star to use new so that you don’t loss events.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...