We have a multisite distributive environment in which we are experiencing performance issue like slowness while running Splunk reports, dashboards and ad-hoc searches in later half of the day, while it is running good in first half. We are looking to use Splunk as 24x7 but its like the first 12hrs it is running good but next 12hrs the performance gets degraded and gets worse.
Can you someone please tell what are the things/performance measures I can check in order to find the issue. One of the reason I believe is regular patching of servers during that hour. How can I investigate something from Splunk end?
Per George, an expert on Splunk multisite deployments at Rational Cyber, "It seems like their WAN is saturated by replication. They should look at queue fullness and at tcp out connections for replication."