Installation

Considerations regarding system-wide resource limits on *nix systems and data segment size (ulimit -d)

yeahnah
Motivator

Hi Splunk Admins

Just looking for some advice around setting the data segment size (ulimit -d) in Splunk, on a Linux  server (RHEL). 

Older documentation (v7.3) recommended setting this value to basically be an unlimited size, with a Kibibyte value of 1073741824 or ~1TB. 
https://docs.splunk.com/Documentation/Splunk/7.3.8/Installation/Systemrequirements#Considerations_re...

Data segment sizeulimit -d1073741824


I see the v8.x documentation has now changed the data segment size recommendation to be more a general guideline, with an 8GB example.

https://docs.splunk.com/Documentation/Splunk/latest/Installation/Systemrequirements#Considerations_r...

Data segment sizeulimit -dThe maximum RAM you want Splunk Enterprise to allocate in kilobytes. For example, 8GB is 8000000. 


It appears Splunk do not really have a strong opinion on a minimum size now either.  I think on RHEV Linux, the data segment size just defaults to unlimited anyway, or at least on our VM servers it does.

I don't believe setting this value alone helps protect Splunk from excessive memory use either.  From what I can tell with googling about data segments, if it was indeed set to a value, then it does not even need to be set to an excessively large value.  Happy to admit I'm no expert though.

Anyway, just wondering if anyone has some experience with setting this value in their environments, or even a view if this data segment size limit even really needs to be set at all - on Linux at least.

Labels (2)
Tags (1)
0 Karma
1 Solution

yeahnah
Motivator

Will answer my own question.

I raised a case with Splunk and they basically came back and said they set their Linux servers to be unlimited for the data segment size value (ulimit -d unlimited).

 

# vi /etc/security/limits.conf
...<snip>...

# Data segment size: ulimit -d
splunk soft data unlimited
splunk hard data unlimited

 

Note, if using systemd then will be set under  /etc/systemd/system/Splunkd.service - refer to doc links in question section.

Not sure why Splunk docs don't just specify the same unlimited value, as the current recommendation is vague and more confusing than useful.

View solution in original post

Tags (1)

yeahnah
Motivator

Will answer my own question.

I raised a case with Splunk and they basically came back and said they set their Linux servers to be unlimited for the data segment size value (ulimit -d unlimited).

 

# vi /etc/security/limits.conf
...<snip>...

# Data segment size: ulimit -d
splunk soft data unlimited
splunk hard data unlimited

 

Note, if using systemd then will be set under  /etc/systemd/system/Splunkd.service - refer to doc links in question section.

Not sure why Splunk docs don't just specify the same unlimited value, as the current recommendation is vague and more confusing than useful.

Tags (1)
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...