Monitoring Splunk

Audit.log restart_splunkd- What produces these messages, and how can I tell if splunkd restarted?

hartsoftware
Engager

I'm seeing many action=restart_splunkd messages from my "_audit" index. I can tell from my processor status that splunkd is not restarting, yet I'm receiving these messages in my _audit index. Can someone help me understand what produces these messages? Also, how can I tell when splunkd actually did restart?

Thanks.

Labels (1)
Tags (2)

som_shekhar
New Member

Hi ,I see this noise in Splunk 8.0.1 also.

0 Karma

andrewtrobec
Motivator

Splunk 8.0.5 too.

0 Karma

araitz
Splunk Employee
Splunk Employee

This is some unfortunate noise from the audit handler. In the future, we hope to improve the audit logging. Genti's answer is correct regarding detecting actual shut downs.

ckurtz
Path Finder

Occurring in 5.0.4, too. Always nice to see the official answer from Genti! (He was here last week helping us)

0 Karma

the_wolverine
Champion

It is still occurring in version 5.0.3.

0 Karma

Genti
Splunk Employee
Splunk Employee

Yeap, 2 more bugs submitted regarding the above

0 Karma

Genti
Splunk Employee
Splunk Employee

Actually, if you notice audit.log will have this message logged every minute, and sometimes more then once per minute. (ie. it sends the action twice - or at least logs it twice)
For real splunkd restart check your splunkd.log (located at /spluhome/var/log/splunk/) for messages like:

10-21-2010 14:40:17.044 INFO  loader - Splunkd starting (build 82143).

and

10-21-2010 14:40:13.029 INFO  ShutdownHandler - Shutdown complete in 2125.5 milliseconds

wandrilleD
Engager

It looks like it's still occuring in newer versions, we are currently in 6.4 and still the same problem.

My question is, with your solution above, it's not possible to track which user did launch the restart?

0 Karma

samsplunks
Explorer

Fast forward to 2019, Splunk 7, the bug is still happening.

One dashboard queries and evals action="restart_splunkd" which causes an Audit:[timestamp=XXX, user=XXX, action=restart_splunkd, info=granted][n/a] log to appear in the _audit index with an audittrail sourcetype (everytime the dashboad is reloaded).

0 Karma

JosephHobbs
Path Finder

Almost 2023 in Splunk 9.x and it's still an issue...

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...