How do I collect basic performance data from my Windows systems?

[sloshburch]

I'm a Splunk administrator, not a Windows administrator, so my Windows knowledge is limited. Nonetheless, a many teams can benefit from having Windows performance data in Splunk. Is there a best practice for collecting basic performance data from Windows endpoints?

[woodcock]

Although everything @SloshBurch said is true, he works for Splunk so his license is free. For those of us who fave to pay for license, there is a little-known FREE alternative called Platform Introspection Forwarder that can be easily enabled and gets you the basics and all that it will cost you is disk space. Do be aware that by default, all the Splunk _* indices have a retention of only 30-days so you might like to extend that if you go this route. One benefit is that you get normalized, platform-agnostic data from ALL forwarders, not just Windows ones:

https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/ConfigurePIF#Enable_the_introspe...

[sloshburch · Accepted solution]

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

If you're new to collecting Windows performance data from endpoints with Splunk, then review Monitor Windows performance in the Getting Data In Manual. This topic has everything you need to understand Splunk configuration details in this post.

Monitor performance data inputs

The Monitor Windows performance documentation lists many data points Splunk can collect. Since you're not a Windows expert, start with percentage-based data for the CPU, storage, memory, and network to identify baseline performance of the endpoints.

Implementation

Use perfmon and WinHostMon data inputs to collect data samples to measure the CPU, storage, memory, and network. An excellent way to implement this is with to configure local Windows performance monitoring with Splunk Web. Alternatively, those interested in Deploying and Using the Splunk Add-on for Windows may prefer to Configure local Windows performance monitoring with configuration files. If you're unfamiliar with the add-on, see our related post Is it a best practice to use the Splunk Add-on for Microsoft Windows?

You'll create the following simplified configuration regardless of your implementation method. Those using Splunk Web can use the stanza label Disk, OperatingSystem, Network, or CPU as the Collection Name for the respective input. Do not include the preceding text of WinHostMon or perfmon or the brackets since Splunk Web can generate that for you in the inputs.conf configuration file. Where applicable, select the type or counter values as detailed in the configuration template below. Those using the configuration file approach can copy and paste the configuration template from below into the appropriate inputs.conf file such as Splunk_TA_windows-mine/local/inputs.conf.

#Memory Free: %
[WinHostMon://OperatingSystem]
    interval = 60
    #index = 
    type = OperatingSystem
    disabled = false

#Network Utilization: Total packets/second (in/out)
[perfmon://Network]
    useEnglishOnly = true
    mode = single
    interval = 60
    #index = 
    counters = Packets/sec; Bytes Total/sec
    disabled = false

#CPU Utilization: %
[perfmon://CPU]
    useEnglishOnly = true
    mode = single
    interval = 60
    #index = 
    counters = % Processor Time
    disabled = false

#Storage Free Space: %
[perfmon://LogicalDisk]
    useEnglishOnly = true
    mode = single
    interval = 60
    #index = 
    counters = % Free Space
    disabled = false

Explanation of settings

There are two standard settings and two input-specific settings to be familiar with. The standard settings are common across most Splunk input types. They are interval and index.

• interval, called Polling Interval within Splunk Web, allows you to increase or decrease your sampling needs. A higher value represents a longer interval between samples and causes a decrease in sampling. The inverse is true as well. F requent sampling provides greater detail but requires more indexing and therefore consumes more storage.
• index corresponds to the target index where the performance data is available for search. Different indexes have different retention and read access capabilities. Learn more in the Managing Indexers and Clusters of Indexers documentation. You can specify the index in Splunk Web or by uncommenting (removing the leading hash #) the index setting in the inputs.conf configuration file you created in this activity. See Specify input settings for details.

The two settings unique to interval and index are useEnglishOnly and mode. The Configure local Windows performance monitoring with configuration files documentation explains these settings and many others.

• useEnglishOnly is an attribute specific to the Performance Monitor, or perfmon input. Although it seems convenient to generate performance information in your native non-English language, the use of this attribute limits the data's applicability to other apps such as Splunk Enterprise Security and IT Service Intelligence. That's because many apps knowledge objects are designed for specific field names or text string matching that are defined only in English. For more information. See the On non-English installations, the useEnglishOnly attribute has usage limitations within the Monitor Windows performance documentation.
• mode is an attribute used by the Performance Monitor, or perfmon input but also available with the WinNetMon input. The default in Splunk is single but all perfmon stanzas in the Splunk® Add-on for Windows are defined to use a mode value of multikv within the default/inputs.conf. Using multikv may be less indexing volume, but be careful because some apps, like Splunk® App for Infrastructure require the use of single.

Do more

If you'd like to learn more, the Splunk ITSI Modules documentation has an excellent topic on how to Configure the Splunk Add-on for Microsoft Windows to collect data and send to your Splunk deployment which documents how to create the local inputs.conf file and additional performance monitoring data inputs.

When the basic event data for the Windows systems are available in Splunk, check out our post What are the best practice searches for Server & OS monitoring? to see searches that can span both Windows and UNIX data.

While slightly more complex, collecting performance data as metric data is more effective than as event data (not to be confused with Windows event logs). Check out Splunk® App for Infrastructure to accelerate and simplify your metrics-based implementation. Once you're ready to implement, consider your migration with our post What is the best way to migrate Windows performance monitoring from event-based to metrics-based dat...

[mrccasi]

Hi,

I know this is an old thread but i just have a question regarding the interval in perfmon. Do you have an idea on how can i get a data every minute? Currently according to the user, the usage data was collected every 15mins and they want to have it every 1 min. Is there any way to do it? Thank you in advance.

[andreasz]

interval = 60

[mrccasi]

Thank you @andreasz