Getting Data In

Resolving Splunk-Perfmon crash errors in forwarder 7.2.9?

wyfwa4
Communicator

Since we upgrades our UF to v7.2.9, we are seeing lots of application crash errors in the application event log on our hosts. This is happening on large volumes of hosts. Initially I thought it may be a specific counter, but it occurs when the Splunk-Perfmon.exe process is running, even if no perfmon collection is occurring. I don't see any errors in Splunk itself and the Splunk-Perfmon process itself keeps running and sending data. Looking into these errors, there seems to be some suggestion this is related to "data execution prevention" which is blocking Splunk trying to run code in data memory (error include code c0000005 which is an access denied error) , but I have not been able to confirm this. servers previously running v6 did not show this error, only when upgraded did the error start to appear.

example error below

SourceName=Windows Error Reporting
EventCode=1001
EventType=4
Type=Information
ComputerName=xxxxxxxxxxxx
TaskCategory=The operation completed successfully.
OpCode=Info
RecordNumber=230239
Keywords=Classic
Message=Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0

Problem signature:
P1: splunk-perfmon.exe
P2: 1794.2305.24028.63924
P3: 5ddcfc22
P4: splunk-perfmon.exe
P5: 1794.2305.24028.63924
P6: 5ddcfc22
P7: c0000005
P8: 00000000005bc5d8
P9:
P10:

Attached files:

These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_splunk-perfmon.e_2f9ed6fb118b57ac0e734f67ff573c73ad1654a_64da0b14_48835327

Labels (1)
0 Karma

avisram_takeda
New Member

We are also experiencing the same issue on Windows Server 2019 with UF 7.3.71. Any updates to this?

0 Karma

nunoaragao
Explorer

I can see the exact same issue with forwarders +8, but only on Win Servers 2019. Has anyone found a fix ?

0 Karma

mwehle_splunk
Splunk Employee
Splunk Employee

Did you happen to find a solution to this? I'm encountering a similar issue.

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...