Getting Data In

Large WMI environment questions

Jason
Motivator

This question is helpful, but I have a client who needs more detail on a WMI-polling environment. Ideally a conference call with an architect. Specifically:

  • What is your largest known WMI-polling architecture?
  • What server in a search head - multiple Indexers - multiple LFs and collectors should do the WMI collection?
  • Is ~120 servers a best practice for number of polled per 16GB Splunk server?
  • What is an appropriate polling frequency? They want as close to real-time alerts as possible.
  • What is the effect on the polled machine?
  • Is load balancing possible, in case LF1 is down, that LF2 can handle the polling?

Let me know if I should break this out into multiple questions on here. Thanks!

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee
  • I don't know
  • A forwarder, or multiple forwarders
  • No. Number of servers is extremely approximate. The limitation is the number of WMI connections that a server can handle, and each query for a server requires a separate connection. So if you're issuing 5 queries, you can only collect from 1/5 as many servers. Furthermore, if you're collecting event logs rather than performance stats via WMI, it will be even more costly since every query must bring back the entire event log on each poll. It is best practice not to collect event logs via WMI, but instead to forward them, either with Splunk forwarders, syslog agents, or Windows event log forwarding.
  • As often as they can tolerate, keeping in mind this also is directly proportional to indexed volume. "As close to real-time as possible" of course comes at a cost that most people are in fact unwilling to bear.
  • Not that much, actually.
  • Yes, but not through pure native Splunk functionality.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee
  • I don't know
  • A forwarder, or multiple forwarders
  • No. Number of servers is extremely approximate. The limitation is the number of WMI connections that a server can handle, and each query for a server requires a separate connection. So if you're issuing 5 queries, you can only collect from 1/5 as many servers. Furthermore, if you're collecting event logs rather than performance stats via WMI, it will be even more costly since every query must bring back the entire event log on each poll. It is best practice not to collect event logs via WMI, but instead to forward them, either with Splunk forwarders, syslog agents, or Windows event log forwarding.
  • As often as they can tolerate, keeping in mind this also is directly proportional to indexed volume. "As close to real-time as possible" of course comes at a cost that most people are in fact unwilling to bear.
  • Not that much, actually.
  • Yes, but not through pure native Splunk functionality.
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...