So I was changing the server time to test another component and couldn't tell if it was the other component that was breaking the DepMon or the time difference. I found an alternative way to test the other component and sure enough, DepMon sees it as active and sending data every minute or so. So ajaitz is right, moving the time into the future on a server hosting a UF will break the communication. It both stopped indexing and wasn't reporting in. Changed the time back, and it picked right back from where it left off. Thank goodness it was reversible! ... View more

Definitely you are indexing historical logs, that are still hot. If this is a timespan issue, a solution to have then roll faster is to define a shorter time range limit for the events of a hot bucket. see maxHotSpanSecs http://docs.splunk.com/Documentation/Splunk/latest/admin/Indexesconf ... View more

Ran into the same error when updating roles/users. I'm going through SSO which occasionally gives me 50x errors if I'm navigating on both my search-head and indexer, and that's when this JSON error seems to crop up. I log off and log back in (no web or daemon restarts) and this issue's resolved. Not sure what the exact cause is or of a better, more permanent solution than "Logoff/Login"... ... View more

There is now official documentation on this: https://docs.splunk.com/Documentation/Splunk/8.0.3/Admin/RunSplunkassystemdservice ... View more