- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Re: Log Monitor Stops from 30-09-2020 to 01-10-202...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have two monitored logs for which no new events are being collected. The Splunk logs don't show any (new) issues or errors - although I did spend quite some time trying to understand if the encoding is the problem. Seing many: "Using charset UTF-16LE, as the monitor is believed over the raw text which may be UTF-8". But these appear for logs that are working and logs that are not.
More digging shows the logs stopped being collected *exactly* after midnight. The first two were collected and the second two were not.
[I 06/00000040/T06BC/P0AA0] 30-09-20 23:47:13 - Client Rules: rule 'Internal Access Policy' matched.
[I 06/0000000A/T06BC/P0AA0] 30-09-20 23:47:13 -Server RAS-SH3:3389 is available
[I 0E/00000000/T1920/P0AA0] 01-10-20 00:15:28 - Session login for userwas successful.
[I 06/00000040/T1920/P0AA0] 01-10-20 00:15:28 - Client Rules: rule 'External Copy and Paste and Printer
I'm guessing I'll need to use the props.conf to set the TIME_FORMAT for these particular files in the app that is collecting them. I've not had to dig this deep into Splunk props before. Wishing myself luck...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You must add TIME_FORMAT to props.conf https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Propsconf with correct parameters. Here it should be %d-%m-%y %H:%M:%S
The easiest way to test these is your dev instance with Settings - Add data - monitor - files and directories. Then just test parameters with correct sourcetype with your sample data.
When it’s working, just copy props.conf and in needed also transforms.conf to your production environment.
r. Ismo
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
the debug info shows the file being monitored without errors.
closer check within the splunk console and parsing the log source shows the logs 01-10-20 getting inserted with the Jan 01 logs.
So having never verifed that these logs were getting parsed correctly in the first place I now see that I've got a mess.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You must add TIME_FORMAT to props.conf https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Propsconf with correct parameters. Here it should be %d-%m-%y %H:%M:%S
The easiest way to test these is your dev instance with Settings - Add data - monitor - files and directories. Then just test parameters with correct sourcetype with your sample data.
When it’s working, just copy props.conf and in needed also transforms.conf to your production environment.
r. Ismo
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes - that was indeed what was needed.
It didn't seem to work initially - after restarting the Splunk server and after restarting the Splunk UF agent on the client. But checking back a day later it was correctly parsing. Most likely I was impatient.
Thank you.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That didn't work...
Moving on to seeing what setting DEBUG in log.cfg shows:
category.TailingProcessor=INFO
category.WatchedFile=INFO
category.ArchiveProcessor=INFO
category.TailReader=INFO
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
Routing logs with Splunk OTel Collector for Kubernetes
