Monitoring Splunk

Why is Splunk is crashing on my AIX system and getting "bad allocation" errors in the splunkd.log?

kserra_splunk
Splunk Employee
Splunk Employee

I just installed splunk on my AIX system and am seeing lots of crashes in the main tailing thread. Additionally I see lots of error messages in the splunkd.log about bad allocation

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

Are there any known configuration settings for AIX that need to be set ? I have splunk installed on non AIX system and it is functioning normally on those systems.

1 Solution

jreuter_splunk
Splunk Employee
Splunk Employee

You should check your data segment size , ulimit -d to make sure that this is set inline with what splunk asks for. By default on AIX systems this is set too low and it can create issues for splunk. Usually when this happens you will see lots of bad allocation error messages in the logs that look like the following:

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

The data segment size (ulimit -d). With Splunk 4.2+, increase the value to at least 1 GB = 1073741824 bytes.

http://docs.splunk.com/Documentation/Splunk/6.1.3/Troubleshooting/ulimitErrors

View solution in original post

jreuter_splunk
Splunk Employee
Splunk Employee

You should check your data segment size , ulimit -d to make sure that this is set inline with what splunk asks for. By default on AIX systems this is set too low and it can create issues for splunk. Usually when this happens you will see lots of bad allocation error messages in the logs that look like the following:

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

The data segment size (ulimit -d). With Splunk 4.2+, increase the value to at least 1 GB = 1073741824 bytes.

http://docs.splunk.com/Documentation/Splunk/6.1.3/Troubleshooting/ulimitErrors

splunkIT
Splunk Employee
Splunk Employee

Recommendation: Minimum of 1G for Data segment (ulimit -d) and for Resident memory (ulimit -m)

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...