Since we've upgraded to 7.0 we're seeing this particular error show up in the logs:
10-17-2017 11:30:30.772 -0600 ERROR PipelineComponent - Monotonic time source didn't increase; is it stuck?
We weren't able to find much information regarding this error online and wanted to poll the audience to see if anyone has encountered this as well.
I had the same question and I opened a Splunk case. This is the response:
"This is an error we have come across with some of our Windows customers, and seems more common of virtualized instances. The splunk process will periodically check the time of the OS system and will show this error if there is a difference (~15 ms) as an indication of the time progress internally. This is really an internal ERROR that should not be reported.
Reference: GetTickCount64 function https://docs.microsoft.com/en-gb/windows/win32/api/sysinfoapi/nf-sysinfoapi-gettickcount64
This issue is currently fixed in version 8.0.0, and if you would like to stop this error from occurring, you will need to look into upgrading to 8.0, otherwise, you can ignore this error message."
I have this error on one heavy forwarder but not the other, all pulling the same configurations? So the datasources are the same on each environment, but only one throws this message followed by:
WARN TcpOutputProc - Tcpout Processor: The TCP output processor has paused the data flow. Forwarding to output group splunkcloud has been blocked for 39750 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.