All Apps and Add-ons

Why is forwarding Windows performance logs not working?

mochocki
Explorer

I have two windows serwers (srv_iis/srv_sql) infrastructure with indexer deployed on srv_iis and forwarder deployed on srv_sql. What I want to achieve is to forward performance counters from srv_sql server to srv_iis.
Facts:
- indexing on the indexer server (srv_iis) works fine
- forwarding event log srv_sql -> srv_iis works fine
- forwarding performance counters the same way is NOT WORKING AT ALL
- WMI is not an option since this is not AD setup (just a workgroup)
- network communication between servers is ok (telnet on mngmt port works fine)
- My Splunk version is 6.5.2
- I tried to deploy this docs.splunk.com/Documentation/MSApp/latest/MSInfra/AbouttheSplunkAppforMSInfrastructure and it is not working as well (perfmon index on the indexer is empty)

inputs.conf from forwarder:

[default]
host = srv_sql

[WinEventLog://Application] <---- this works fine
disabled = 0
index = perf

[perfmon://LocalMainMemory]
interval = 5
object = Memory
counters = Committed Bytes; Available Bytes; % Committed Bytes In Use
disabled = 0
index = perf

[perfmon://Available Memory]
counters = *
interval = 10
object = Memory
index = perf

outputs.fonf from forwarder:

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = srv_iis:9997

[tcpout-server://srv_iis:9997]
0 Karma
1 Solution

adonio
SplunkTrust
SplunkTrust

Please relay on Windows TA pre built inputs for example:
[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Output/sec; Pool Paged Bytes; Pool Nonpaged Bytes; Page Writes/sec; Pool Paged Allocs; Pool Nonpaged Allocs; Free System Page Table Entries; Cache Bytes; Cache Bytes Peak; Pool Paged Resident Bytes; System Code Total Bytes; System Code Resident Bytes; System Driver Total Bytes; System Driver Resident Bytes; System Cache Resident Bytes; % Committed Bytes In Use; Available KBytes; Available MBytes; Transition Pages RePurposed/sec; Free & Zero Page List Bytes; Modified Page List Bytes; Standby Cache Reserve Bytes; Standby Cache Normal Priority Bytes; Standby Cache Core Bytes; Long-Term Average Standby Cache Lifetime (s)
disabled = 0
interval = 10
object = Memory
useEnglishOnly=true
index = perfmon (or index=perf)
download the TA here: https://splunkbase.splunk.com/app/742/
navigate to ...\apps\splunk_TA_Windows\default and check all inputs at inupts.conf
create a local directory and copy the needed inputs.
modify disabled = 1 to disabled = 0 for the inputs you wish to enable
Cheers

View solution in original post

adonio
SplunkTrust
SplunkTrust

Please relay on Windows TA pre built inputs for example:
[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Output/sec; Pool Paged Bytes; Pool Nonpaged Bytes; Page Writes/sec; Pool Paged Allocs; Pool Nonpaged Allocs; Free System Page Table Entries; Cache Bytes; Cache Bytes Peak; Pool Paged Resident Bytes; System Code Total Bytes; System Code Resident Bytes; System Driver Total Bytes; System Driver Resident Bytes; System Cache Resident Bytes; % Committed Bytes In Use; Available KBytes; Available MBytes; Transition Pages RePurposed/sec; Free & Zero Page List Bytes; Modified Page List Bytes; Standby Cache Reserve Bytes; Standby Cache Normal Priority Bytes; Standby Cache Core Bytes; Long-Term Average Standby Cache Lifetime (s)
disabled = 0
interval = 10
object = Memory
useEnglishOnly=true
index = perfmon (or index=perf)
download the TA here: https://splunkbase.splunk.com/app/742/
navigate to ...\apps\splunk_TA_Windows\default and check all inputs at inupts.conf
create a local directory and copy the needed inputs.
modify disabled = 1 to disabled = 0 for the inputs you wish to enable
Cheers

View solution in original post

adonio
SplunkTrust
SplunkTrust

Hi mochocki,
can you verify its index=perfmon and not index=perf as shows in your code?
or you created that index for the data?
also, are you usinf the windows TA? https://splunkbase.splunk.com/app/742/

0 Karma

mochocki
Explorer

Hi,
Index perfmon comes from MSApp. Index perf comes from my configuration. Both do not contain any performance entries.

0 Karma

adonio
SplunkTrust
SplunkTrust

are you using the rebuilt perfmon inputs from the TA? can yuo try and place this in your inputs.conf and check?
[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Output/sec; Pool Paged Bytes; Pool Nonpaged Bytes; Page Writes/sec; Pool Paged Allocs; Pool Nonpaged Allocs; Free System Page Table Entries; Cache Bytes; Cache Bytes Peak; Pool Paged Resident Bytes; System Code Total Bytes; System Code Resident Bytes; System Driver Total Bytes; System Driver Resident Bytes; System Cache Resident Bytes; % Committed Bytes In Use; Available KBytes; Available MBytes; Transition Pages RePurposed/sec; Free & Zero Page List Bytes; Modified Page List Bytes; Standby Cache Reserve Bytes; Standby Cache Normal Priority Bytes; Standby Cache Core Bytes; Long-Term Average Standby Cache Lifetime (s)
disabled = 0
interval = 10
object = Memory
useEnglishOnly=true
index = perfmon (or index=perf)

0 Karma

mochocki
Explorer

It works! Thank you!
Still do not understand what was wrong. The only difference I see is useEnglishOnly=true.
My Windows locale is Polish - is that the problem?

0 Karma

adonio
SplunkTrust
SplunkTrust

there is also a difference in the stanza [perfmon://LocalMainMemory] - yours
compare to [perfmon://Memory] - prebuilt Windows TA
will place in the answer section
cheers

0 Karma