What would cause a Fatal thread error in thread typing found in the Splunk crash log?

[j4adam]

Hello everyone,

Over a ~26 hour period I am seeing several fatal errors regarding the typing thread. I've done some research and there is very little information about threads to be found online in regards to how splunk uses them. I am making the assumption that the typing thread is the thread in charge of the typing queue.

The actual error is found in the splunk crash log:

Fatal thread error: pthread_mutex_lock: Invalid argument: 78 threads active in typing

(followed by several more with varying number of active threads, all less than 78)

Dying on fatal thread error 22 from pthread_mutex_lock, thread typing

As far as I remember, mutex are usedto control the availability of threads on a system to prevent race conditions in multithreading. This leads me to believe that perhaps the system is running out of threads when trying to use the typing queue. If this is true, a few things confuse me:

1. If this is running out of threads, why would the active thread count fluctuate on the fatal attempt?
2. If this is the case, why would splunk not have a graceful way of handling this?

These questions I have with my conclusion lead me to believe there is something I'm missing. (My knowledge of mutex is from years ago, so I could very well be wrong).

Any insight would be great. The issue hasn't resurfaced in ~20 days and has only happened once, but knowing the cause would help prevent it in the future.

Thank you.

[Ciccio]

Anyone has solved the problem? Splunk case has a solution? I am talking about " FATAL Thread - Fatal thread error: pthread_mutex_lock: Invalid argument; 66 threads active, in typing" Form me the reboot wasn't a solution. thank You.

[hrawat]

See if this link helps .

[cygnetix]

I was able to resolve this problem by rebooting the server.

[laurent_]

We have the same problem. I will open a support ticket with Splunk.

FATAL Thread - Fatal thread error: pthread_mutex_lock: Invalid argument; 49 threads active, in typing

[tmeader]

We have an active case open on this same issue. Please reference case #377552 when you open your ticket. I'd love the help on getting some action on this, as we're seeing it fairly regularly.