Splunk dropping events intermittently

[surajsplunkd]

Could you please tell me why WinHostMon events are missing intermittently in Splunk?

I dont see any Error in internal logs except Warning and Info.

Thanks in advance

[surajsplunkd]

@ITWhisperer  thanks for your response, Events have actually occurred at all the instances for which there is no event in splunk, I have checked that on the remote server. However, I dont have any evidence which I can share here. This issue has been intermittently happening only on a few hosts out of 16 since the start of this year. In past we have tested by increasing thruput and restarting splunk-winhostmon.exe on the problematic hosts. It was not helpful though. Is there any particular troubleshooting steps, logs or documentation that you could refer me?

I have already gone through this Monitor Windows host information - Splunk Documentation

[ITWhisperer]

The fact that it is intermittent seems to suggest that there may be nothing wrong with your Splunk configuration and that it is more likely to be outside of Splunk.

What else is going on on the hosts at the time?

What else is going on in your network?

Are there other processes which could be delaying the traffic or preventing the events from occurring or being logged?

Apart from doing wider analysis of your environment, you could keep monitoring it to see if you can detect any patterns, e.g. always the same hosts? are they at different patch levels, are they on different parts of the network? do they support different applications? is the user community on those hosts different in some way? is there a correlation to the time of day, day of the week, or other time factor?

[surajsplunkd]

@ITWhisperer  Is there any way in splunk,  I could check if UF has hit the max allowed thruput? I think by default it is 256kbps and so max thruput per day should be 256*60*60*24 =22,118,400 right?

[PickleRick]

Even if you hit the thruput limit, it should not on its own cause event loss - just delay.

[ITWhisperer]

I have no idea about that - someone else might know - you could ask your administrators if the "default" can be increased to see if that helps (or reduced to see if it makes the problem worse - which might prove that this is the issue!). Other than that, is there anything in any of the _internal logs to suggest throughput issues?

[ITWhisperer]

What evidence do you have that events actually occurred during these times?