Monitoring Splunk

Windows Perfmon:Process index fine tuning

ansif
Motivator

Perfmon:Process is about 347,662 events for 2 host last 10 mins which is taking huge space in index. Any suggestion to fine tune this?

Tags (1)
0 Karma
1 Solution

lycollicott
Motivator

This is really an input tuning problem and not an index problem.

When you enable it, Splunk_TA_windows collects almost every perfmon counter in existence and it does it every 10 seconds.

First of all, no one needs to collect all of those counters.
Secondly, every 10 seconds is far too fast for most situations.

The perfmon process counters are turned off by default, so why exactly do you need them? It would take a very compelling reason for me to collect them on my servers.

Here is what the default settings are in etc\apps\Splunk_TA_windows\default\inputs.conf and it collects 27 different counters for each process every 10 seconds:

[perfmon://Process]
counters = % Processor Time; % User Time; % Privileged Time; Virtual Bytes Peak; Virtual Bytes; Page Faults/sec; Working Set Peak; Working Set; Page File Bytes Peak; Page File Bytes; Private Bytes; Thread Count; Priority Base; Elapsed Time; ID Process; Creating Process ID; Pool Paged Bytes; Pool Nonpaged Bytes; Handle Count; IO Read Operations/sec; IO Write Operations/sec; IO Data Operations/sec; IO Other Operations/sec; IO Read Bytes/sec; IO Write Bytes/sec; IO Data Bytes/sec; IO Other Bytes/sec; Working Set - Private
disabled = 1
instances = *
interval = 10
object = Process
useEnglishOnly=true
index = perfmon

If you are being forced against your will to collect process counters then:

  1. decide which ones you really, really need and collect only those
  2. collect them less frequently

View solution in original post

lycollicott
Motivator

This is really an input tuning problem and not an index problem.

When you enable it, Splunk_TA_windows collects almost every perfmon counter in existence and it does it every 10 seconds.

First of all, no one needs to collect all of those counters.
Secondly, every 10 seconds is far too fast for most situations.

The perfmon process counters are turned off by default, so why exactly do you need them? It would take a very compelling reason for me to collect them on my servers.

Here is what the default settings are in etc\apps\Splunk_TA_windows\default\inputs.conf and it collects 27 different counters for each process every 10 seconds:

[perfmon://Process]
counters = % Processor Time; % User Time; % Privileged Time; Virtual Bytes Peak; Virtual Bytes; Page Faults/sec; Working Set Peak; Working Set; Page File Bytes Peak; Page File Bytes; Private Bytes; Thread Count; Priority Base; Elapsed Time; ID Process; Creating Process ID; Pool Paged Bytes; Pool Nonpaged Bytes; Handle Count; IO Read Operations/sec; IO Write Operations/sec; IO Data Operations/sec; IO Other Operations/sec; IO Read Bytes/sec; IO Write Bytes/sec; IO Data Bytes/sec; IO Other Bytes/sec; Working Set - Private
disabled = 1
instances = *
interval = 10
object = Process
useEnglishOnly=true
index = perfmon

If you are being forced against your will to collect process counters then:

  1. decide which ones you really, really need and collect only those
  2. collect them less frequently
Get Updates on the Splunk Community!

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...