All Apps and Add-ons

Splunk AddOn Microsoft Cloudservices Memory Leak?

gazoscreek
Loves-to-Learn Lots

We're seeing severe memory issues with the Splunk AddOn for Microsoft Cloudservices where a single python process can consume well over 200Gb of memory, ultimately being reaped by the OOM Killer.

These are Azure Firewall collections. Rather critical to our logging infrastructure and we have no information as to why this is happening. It doesn't happen to all collections, but does happen to many ... You can see below, one child process consuming > 60Gb ... and Splunkd having grown to 275Gb of virtual memory.

splunk 17311 125 67318812 62146568 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_blob.py
splunk 15910 227 274447324 82630100 splunkd -p 8089 restart

Splunk version 8.0.4.1

Splunk_TA_microsoft-cloudservices version 4.1.2

 

 

Labels (2)
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.