Getting Data In

Why are the ulimits set correctly, not showing in splunk web?

TaraPennington
Loves-to-Learn Lots

I'm working on the initial set up of splunk single instance on prem and I haven't been able to get data in yet. I have installed the universal forwarder on 2 windows servers and installed the add on for windows on those servers. I get this message in the monitoring console.

ulimits.data_segment_size (current / recommended) ulimits.open_files (current / recommended) ulimits.user_processes (current / recommended)
-1 4096 / 64000 47318 / 16000

 

Then when I log onto the Cent OS server and look at ulimits and they are set as the recommended minimum values.

 

ulimits on server.PNG

How can I get the Splunk web to recognize how these settings are set on the server?

Labels (4)
Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @TaraPennington,

Can you try restarting the server? 

https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/ulimitErrors#Set_limits_using_.2...

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I rebooted the server and still seeing the same messages in splunk web. Do I need to also change these settings from the link you sent? 

Set limits using the /etc/systemd configuration files

Tags (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

If Splunk is running under systemd , it will help.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

TaraPennington
Loves-to-Learn Lots

I added these lines at the end of the /etc/security/limits.conf on the root profile, I'm still getting the same message.

TaraPennington_0-1611679553758.png

I didn't configure splunk to run on the systemd, so I didn't add those other settings.

This is how the bottom of the file looks, I'm not sure if these are entered correctly.

TaraPennington_1-1611679710764.png

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

for which user did you setted your ulimits?

You have to se it for te user who runs splunk process (usually root or splunk).

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

It was using the root user account.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @TaraPennington,

did you configured ulimit in /etc/security/limits.conf ?
if not, you have to insert at the end of this file:

root hard nofile 64000
root soft nofile 64000

then exit from the user or restart Splunk.

Ciao.

Giuseppe

0 Karma

TaraPennington
Loves-to-Learn Lots

ulimits setulimits set

I believe I added those two lines to the end of the /etc/security/limits.conf correctly

I saved this and restarted splunk and am still getting the same message about ulimits.

0 Karma

gcusello
SplunkTrust
SplunkTrust

hi @TaraPennington,

to apply the updates, you have to:

  • exit the user,
  • access again,
  • restart Splunk.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...