I have two monitored logs for which no new events are being collected. The Splunk logs don't show any (new) issues or errors - although I did spend quite some time trying to understand if the encoding is the problem. Seing many: "Using charset UTF-16LE, as the monitor is believed over the raw text which may be UTF-8". But these appear for logs that are working and logs that are not.
More digging shows the logs stopped being collected *exactly* after midnight. The first two were collected and the second two were not.
[I 06/00000040/T06BC/P0AA0] 30-09-20 23:47:13 - Client Rules: rule 'Internal Access Policy' matched.
[I 06/0000000A/T06BC/P0AA0] 30-09-20 23:47:13 -Server RAS-SH3:3389 is available
[I 0E/00000000/T1920/P0AA0] 01-10-20 00:15:28 - Session login for userwas successful.
[I 06/00000040/T1920/P0AA0] 01-10-20 00:15:28 - Client Rules: rule 'External Copy and Paste and Printer
I'm guessing I'll need to use the props.conf to set the TIME_FORMAT for these particular files in the app that is collecting them. I've not had to dig this deep into Splunk props before. Wishing myself luck...
the debug info shows the file being monitored without errors.
closer check within the splunk console and parsing the log source shows the logs 01-10-20 getting inserted with the Jan 01 logs.
So having never verifed that these logs were getting parsed correctly in the first place I now see that I've got a mess.
Yes - that was indeed what was needed.
It didn't seem to work initially - after restarting the Splunk server and after restarting the Splunk UF agent on the client. But checking back a day later it was correctly parsing. Most likely I was impatient.
That didn't work...
Moving on to seeing what setting DEBUG in log.cfg shows: