Getting Data In

Why are the ulimits set correctly, not showing in splunk web?

TaraPennington
Loves-to-Learn Lots

I'm working on the initial set up of splunk single instance on prem and I haven't been able to get data in yet. I have installed the universal forwarder on 2 windows servers and installed the add on for windows on those servers. I get this message in the monitoring console.

ulimits.data_segment_size (current / recommended) ulimits.open_files (current / recommended) ulimits.user_processes (current / recommended)
-1 4096 / 64000 47318 / 16000

 

Then when I log onto the Cent OS server and look at ulimits and they are set as the recommended minimum values.

 

ulimits on server.PNG

How can I get the Splunk web to recognize how these settings are set on the server?

Labels (3)
Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @TaraPennington,

Can you try restarting the server? 

https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/ulimitErrors#Set_limits_using_.2...

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I rebooted the server and still seeing the same messages in splunk web. Do I need to also change these settings from the link you sent? 

Set limits using the /etc/systemd configuration files

Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

If Splunk is running under systemd , it will help.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I added these lines at the end of the /etc/security/limits.conf on the root profile, I'm still getting the same message.

TaraPennington_0-1611679553758.png

I didn't configure splunk to run on the systemd, so I didn't add those other settings.

This is how the bottom of the file looks, I'm not sure if these are entered correctly.

TaraPennington_1-1611679710764.png

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

for which user did you setted your ulimits?

You have to se it for te user who runs splunk process (usually root or splunk).

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

It was using the root user account.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

did you configured ulimit in /etc/security/limits.conf ?
if not, you have to insert at the end of this file:

root hard nofile 64000
root soft nofile 64000

then exit from the user or restart Splunk.

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

ulimits setulimits set

I believe I added those two lines to the end of the /etc/security/limits.conf correctly

I saved this and restarted splunk and am still getting the same message about ulimits.

0 Karma

gcusello
SplunkTrust
SplunkTrust

hi @TaraPennington,

to apply the updates, you have to:

  • exit the user,
  • access again,
  • restart Splunk.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 3)

Welcome back to Splunk Classroom Chronicles, our ongoing blog series that pulls back the curtain on Splunk ...

Operationalizing TDIR: Building a More Resilient, Scalable SOC

Optimizing SOC workflows with a unified, risk-based approach to Threat Detection, Investigation, and Response ...

Almost Too Eventful Assurance: Part 1

Modern IT and Network teams still struggle with too many alerts and isolating issues before they are notified. ...