Splunk Add-on for Microsoft Windows: How to limit the amount of Perfmon events?

[EdgarAllenProse]

I have several machines being monitored with perfmon, and I am struggling to figure out how to limit the amount of logs coming in.

Here is the perfmon section of inputs.conf in Splunk Add-on for Microsoft Windows:

###### Splunk 5.0+ Performance Counters ######
## CPU
[perfmon://CPU]
counters = % Processor Time; % User Time; % Privileged Time; Interrupts/sec; % DPC Time; % Interrupt Time; DPCs Queued/sec; DPC Rate; % Idle Time; % C1 Time; % C2 Time; % C3 Time; C1 Transitions/sec; C2 Transiti$
disabled = 0
instances = *
interval = 30
object = Processor
useEnglishOnly=true
index = perfmon

## Logical Disk
[perfmon://LogicalDisk]
counters = % Free Space; Free Megabytes; Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk $
disabled = 0
instances = *
interval = 30
object = LogicalDisk
useEnglishOnly=true
index = perfmon

## Physical Disk
[perfmon://PhysicalDisk]
counters = Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk sec/Transfer; Avg. Disk sec/Re$
disabled = 1
instances = *
interval = 10
object = PhysicalDisk
useEnglishOnly=true
index = perfmon

## Memory
[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Ou$
disabled = 0
interval = 30
object = Memory
useEnglishOnly=true
index = perfmon

So in an attempt to ease back on the amount of logs, all that seems to be getting accomplished is limiting the frequency at which I receive them.

• Instead of getting 5 logs every second, I am getting ~9000 logs every 30 minutes.

Instead of having Splunk poll the data every 30 minutes, is there a way to have it only grab the current state logs? I don't need every single perfmon log every single second, if I can have splunk check every 5 minutes and just grab the most recent log per host per sourcetype (specific to the perfmon), that would be extremely helpful.

I'm aware of the interval being at 30 minutes, I was tweaking it trying to see if I could limit ingestion.

• Instead of getting 5 logs per second for each host, I want to get 5 logs per 5 minutes for each host.

Thanks for your time and any help/advice is greatly appreciated.

[jkat54]

Set your intervals to 300 (60 seconds/minute x 5 minutes = 300 seconds).

[somesoni2 · Accepted solution]

The interval is in seconds, so you would be getting

[perfmon://CPU] - 60 entries per 30 min per server
[perfmon://LogicalDisk] - 60 entries per 30 min per disk per server
[perfmon://PhysicalDisk] - 180 entries per 30 min per disk per server
[perfmon://Memory] - 60 entries per 30 min per server

If you want a snapshot of every 5 min, set the interval to 300 (seconds) OR to a cron that run every 5 mins. Like this

###### Splunk 5.0+ Performance Counters ######
 ## CPU
 [perfmon://CPU]
 counters = % Processor Time; % User Time; % Privileged Time; Interrupts/sec; % DPC Time; % Interrupt Time; DPCs Queued/sec; DPC Rate; % Idle Time; % C1 Time; % C2 Time; % C3 Time; C1 Transitions/sec; C2 Transiti$
 disabled = 0
 instances = *
 interval = 4-59/5 * * * *
 object = Processor
 useEnglishOnly=true
 index = perfmon

 ## Logical Disk
 [perfmon://LogicalDisk]
 counters = % Free Space; Free Megabytes; Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk $
 disabled = 0
 instances = *
 interval = 3-59/5 * * * *
 object = LogicalDisk
 useEnglishOnly=true
 index = perfmon

 ## Physical Disk
 [perfmon://PhysicalDisk]
 counters = Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk sec/Transfer; Avg. Disk sec/Re$
 disabled = 1
 instances = *
 interval = 2-59/5 * * * *
 object = PhysicalDisk
 useEnglishOnly=true
 index = perfmon

 ## Memory
 [perfmon://Memory]
 counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Ou$
 disabled = 0
 interval = 1-59/5 * * * *
 object = Memory
 useEnglishOnly=true
 index = perfmon

[EdgarAllenProse]

Thanks for the explanation, changing interval worked. I also realized I was seeing so many logs because of the excessive amount of counters.
So changing
interval=30 (~90 logs every 30 seconds)
to
interval=300 (~90 logs every 300 seconds)

The counters creating so many separate logs made me think setting the interval to 30 was making no difference in log count. But changing it to the way you advised gives me a lot less over time. Thanks!