Getting Data In

How to monitor Server hung state on both Windows and Linux using Splunk?

ansif
Motivator

Can anyone help me to know the possibility of monitoring server hung state using Splunk?

0 Karma
1 Solution

woodcock
Esteemed Legend

This app was just posted yesterday and probably does everything you need (and if not the author will surely be very responsive):
Unified Forwarder Monitoring App for Splunk: https://splunkbase.splunk.com/app/3805/

View solution in original post

woodcock
Esteemed Legend

This app was just posted yesterday and probably does everything you need (and if not the author will surely be very responsive):
Unified Forwarder Monitoring App for Splunk: https://splunkbase.splunk.com/app/3805/

ansif
Motivator

This is working and it has the module what @niketnilay suggetsed.

Thanks @woodcock and @niketnilay

0 Karma

niketn
Legend

@ansif, we had an issue on our Windows servers which used to hang during scheduled reboot and would not shut down. While servers used to respond to ping request, almost all services including the Remote Login used to stop functioning. Obviously Splunk's splunkd service also used to stop. Which implied that Splunk Universal Forwarder on such Windows machines (Splunk Deployment Client) would stop pinging Deployment Server. We could do either one of the following to check for such events:

1) Check the Deployment Console in DMC to identify the deployment clients which have not pinged recently.

2) Use REST API to get the deployment clients which have not pinged recently (following example if for last 5 min):

| rest /services/deployment/server/clients
| table name lastPhoneHomeTime
| where lastPhoneHomeTime>300

PS: DMC also uses the same REST API: https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTdeploy#deployment.2Fserver.2Fclients

If you are already indexing some stats from these Windows and/or Linux machines(even if Splunk's internal logs from these hosts) there may be several other options as well. Refer to the following Answer thread: https://answers.splunk.com/answers/592278/query-for-splunkd-status.html#answer-593319

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

ansif
Motivator

@niketnilay : Could you please help me with a solution on below:

Our deployment server is at customer site,is there any option to create an inputs.conf definition?

0 Karma

woodcock
Esteemed Legend

Why do you post so many answers as comments, @niketnilay?

0 Karma

niketn
Legend

@woodcock, 😄 for several reasons but I mostly try to keep questions as unanswered so that others can assist with their inputs as well. Other cases when I require further clarification or am not sure of the solution I provide then also I post as a comment to hear back. Keeping as unanswered leaves the question open for others in case the poster responds back and I miss.

This is complete different case though 🙂 Ansif and I work at same company, so I can follow up with him at work 🙂

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...