I have a real time alert set for admin accounts whenever they make a change and create Event code 4738. All client UF are running win 2012r2 . Spunk support was with me one day and we fine delays in index time. some client work within minutes while other take hours later or the next day. Another issue related is listing all Domain Controller and some are missing with the command below.
index=winevents source="WinEventLog:Security" | rare limit=50 host
All Domain Controller should come up as they are all the same hardware, OS, patch level & same UF installed.
Upgrading the UF from 6.2.3 to 6.2.7 did not help.
Also upgrade my Heavy Forwarder to 6.2.7 did not help.
I have no load issues with my index cluster and all system log from the DC index, but 4738 Security logs don't.
How can I fix this? or bet way to debug between UF to HF to Indexers?
First, try this...
earliest=-1h@h latest=+2y index=winevents source="WinEventLog:Security"
| stats max(time) as maxtime count by host
| rename maxtime as _time
Check if any events are future-dated, and check to make sure that every host is present.
I would not use rare
in this situation, since it limits the number of results.