Monitoring Splunk

Why is Splunk is crashing on my AIX system and getting "bad allocation" errors in the splunkd.log?

kserra_splunk
Splunk Employee
Splunk Employee

I just installed splunk on my AIX system and am seeing lots of crashes in the main tailing thread. Additionally I see lots of error messages in the splunkd.log about bad allocation

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

Are there any known configuration settings for AIX that need to be set ? I have splunk installed on non AIX system and it is functioning normally on those systems.

1 Solution

jreuter_splunk
Splunk Employee
Splunk Employee

You should check your data segment size , ulimit -d to make sure that this is set inline with what splunk asks for. By default on AIX systems this is set too low and it can create issues for splunk. Usually when this happens you will see lots of bad allocation error messages in the logs that look like the following:

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

The data segment size (ulimit -d). With Splunk 4.2+, increase the value to at least 1 GB = 1073741824 bytes.

http://docs.splunk.com/Documentation/Splunk/6.1.3/Troubleshooting/ulimitErrors

View solution in original post

jreuter_splunk
Splunk Employee
Splunk Employee

You should check your data segment size , ulimit -d to make sure that this is set inline with what splunk asks for. By default on AIX systems this is set too low and it can create issues for splunk. Usually when this happens you will see lots of bad allocation error messages in the logs that look like the following:

ERROR PropertiesMapConfig - Failed to save stanza /var/adm/sudo.log_Mon_Sep_22_16:37:27_2014_1998275973 to app learned: bad allocation

The data segment size (ulimit -d). With Splunk 4.2+, increase the value to at least 1 GB = 1073741824 bytes.

http://docs.splunk.com/Documentation/Splunk/6.1.3/Troubleshooting/ulimitErrors

View solution in original post

splunkIT
Splunk Employee
Splunk Employee

Recommendation: Minimum of 1G for Data segment (ulimit -d) and for Resident memory (ulimit -m)

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!