Getting Data In

How to stop getting duplicate events from WindowsUpdateLog?

hettervik
Builder

Hi. I have a problem that I'm getting duplicate events from WindowsUpdate.log. I'm pretty sure it's related to the following message I'm getting in the internal logs Logfile truncated while open, original pathname="C:/winupdate/blabla", will begin reading from start. Though, if the logfile really was truncated (all the data in the file deleted), then I wouldn't get duplicate events. It seems to me that Splunk only "thinks" that the log file was truncated, but in reality it's not.

Can anyone explain to me how this "truncating" of the WindowsUpdateLog works, and why it can cause duplicate events in Splunk? Also, if someone could offer a possible solution, that would be nice as well!

0 Karma
1 Solution

hettervik
Builder

Seems like this problem has been addressed in newer versions of the supported Splunk Windows TA.

https://docs.splunk.com/Documentation/AddOns/released/Windows/Configuration#Configure_Windows_Update...

View solution in original post

0 Karma

hettervik
Builder

Seems like this problem has been addressed in newer versions of the supported Splunk Windows TA.

https://docs.splunk.com/Documentation/AddOns/released/Windows/Configuration#Configure_Windows_Update...

0 Karma

DavidHourani
Super Champion

hi @hettervi,

This is an informational message, isn't it ? Doesn't seem like an error to me. It only means that the log file got rotated, if you're getting duplicates it could be because the file is generated with the same data again.

Is there any timestamp in the file or anything that makes the events unique ? You could try to play around with initCrcLength and crcSaltto avoid having the file indexed over and over

0 Karma

hettervik
Builder

Yes. It's not an error message per se. It doesn't necessarily mean that the windowsupdatelog got rotated, and that's the problem I think. For some reason, instead of rotating the logs in a normal fashion, Windows truncate the log file and starts writing it over again, if I understand correctly, but I'm not sure. Though it doesn't make any sense that Windows would generate the same file over again with the same data.

I have timestamps, and they are not unique. That is, I'm positive that the events I'm getting are in fact duplicate events from the same machine. Same timestamp, same everything, except indextime of course.

0 Karma

DavidHourani
Super Champion

so from what you're saying it could be that since Windows is modifying something in the header of the file the rest of the file is getting indexes all over again.

Could you please share your inputs configuration ?

0 Karma

hettervik
Builder

I don't have the input configuration available right now unfortunately, but it's straight from the official Windows TA from Splunk. Is uses a normal monitoring stanza for tailing the log. I think the path is C:\Windows\Logs\WindowsUpdate.

0 Karma

hettervik
Builder

For the record I've also tried using the setting ignoreOlderThan=1h, but it has no effect. I will try using another inputs.conf setting followTail=1. It's kind of a hacky solution, but if it stops duplicate entries I'm happy.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...