All Apps and Add-ons

Using Splunk on t2.micro Linux instance, why does the splunkd service need to be restarted to keep it running and how do I resolve this?

prasasthi001
New Member

Hi,

I have a t2.micro Linux instance running as a Splunk node. The Splunk instance sometimes doesn't pass status checks on AWS. When I stop and restart the instance again, it works. I SSH into the instance and check the status every time I cannot access the home page. It shows that the splunkd is not running. I restart the process and then I can access the Splunk page on port 8000 again. Please help me resolve this issue.

Thank you.
Sai

0 Karma

Jeremiah
Motivator

The t2.micro instance has 1 (burstable) cpu and 1 GB of memory, which barely meet the Splunk minimum hw requirements. How much data are you pushing onto this system? How many users are accessing the UI? It's likely the process is crashing due to resource constraints. There are a couple of ways you can check this.

First, look at /opt/splunk/var/log/splunk and check for crash files. These files indicate the process crashed unexpectedly. If you have a support contract Splunk can use these files to help determine the cause of the crash.

Look at the sourcetype=splunkd log files from your instance at the time of the crash. Are there any errors or warnings that might indicate a problem?

Check the cloudwatch metrics for this instance. How is the CPU utilization? Disk and network IO? If you have the CW agent enabled, check memory utilization. You can also look at detailed host metrics collected by Splunk in the _introspection index. Check the DMC for any indications of resource constraints, especially memory.

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...