Getting Data In

Why are the ulimits set correctly, not showing in splunk web?

TaraPennington
Loves-to-Learn Lots

I'm working on the initial set up of splunk single instance on prem and I haven't been able to get data in yet. I have installed the universal forwarder on 2 windows servers and installed the add on for windows on those servers. I get this message in the monitoring console.

ulimits.data_segment_size (current / recommended) ulimits.open_files (current / recommended) ulimits.user_processes (current / recommended)
-1 4096 / 64000 47318 / 16000

 

Then when I log onto the Cent OS server and look at ulimits and they are set as the recommended minimum values.

 

ulimits on server.PNG

How can I get the Splunk web to recognize how these settings are set on the server?

Labels (4)
Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @TaraPennington,

Can you try restarting the server? 

https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/ulimitErrors#Set_limits_using_.2...

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I rebooted the server and still seeing the same messages in splunk web. Do I need to also change these settings from the link you sent? 

Set limits using the /etc/systemd configuration files

Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

If Splunk is running under systemd , it will help.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I added these lines at the end of the /etc/security/limits.conf on the root profile, I'm still getting the same message.

TaraPennington_0-1611679553758.png

I didn't configure splunk to run on the systemd, so I didn't add those other settings.

This is how the bottom of the file looks, I'm not sure if these are entered correctly.

TaraPennington_1-1611679710764.png

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

for which user did you setted your ulimits?

You have to se it for te user who runs splunk process (usually root or splunk).

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

It was using the root user account.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

did you configured ulimit in /etc/security/limits.conf ?
if not, you have to insert at the end of this file:

root hard nofile 64000
root soft nofile 64000

then exit from the user or restart Splunk.

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

ulimits setulimits set

I believe I added those two lines to the end of the /etc/security/limits.conf correctly

I saved this and restarted splunk and am still getting the same message about ulimits.

0 Karma

gcusello
SplunkTrust
SplunkTrust

hi @TaraPennington,

to apply the updates, you have to:

  • exit the user,
  • access again,
  • restart Splunk.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...