Log File Rotation causing new file to not ingest

[TRobertshaw]

We have a file that is rotated at midnight every night.  The file is renamed and zipped up. 

Sometimes after the log rotation Splunk does not ingest the new file.

There are no errors in the Splunkd log relating to crc or anything along those lines.

A restart of Splunk resolves the issue however we would like to find a more permanent solution.

We are on UF version, 9.0.4.

Appreciate any suggestions you may have

[PickleRick]

Are you sure that the file is _rotated_ (as in renamed and compressed)? Because that behaviour is pretty consistent with the "copytruncate" behaviour of logrotate when the contents of the file are copied out to a new file and the file is truncated afterwards. In such case the file descriptor does not change but Splunk suddenly finds itself after the end of the data so most probably assumes that it had already read all the data there was.

[TRobertshaw]

Is there a way to allow Splunk to refresh and review the new file everytime?

[PickleRick]

You'd have to restart the forwarder service after logrotate. (Because I assume that's what you're using). Just like normally you kill -HUP your syslog daemon.