Dashboards & Visualizations

Splunk Restart Tracking

mbasharat
Builder

Hi folks,

I want to setup a dashboard to track Splunk activities. I need to know how to track who restarted Splunk via both UI and audit logs in Splunk Dashboard?

Thanks in-advance.

Tags (1)
0 Karma
1 Solution

PowerPacked
Builder

Hi @mbasharat

Please use these searches

index=_audit action=restart_splunkd | stats c by user
index=_audit action=splunkStarting
index=_audit action=splunkShuttingDown

for UI restart its writes in Splunkd_stdout.log

index=_internal sourcetype=splunkd_stdout

Thanks

View solution in original post

bandit
Motivator

Splunk Restart Dashboard

<form theme="dark">
  <label>Splunk Restarts</label>
  <fieldset submitButton="false">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="host_include_pattern" searchWhenChanged="true">
      <label>Host Include Pattern (host1,host2)</label>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="host_exclude_pattern" searchWhenChanged="true">
      <label>Host Exclude Pattern</label>
      <default>null</default>
      <prefix>NOT host="*</prefix>
      <suffix>*"</suffix>
    </input>
  </fieldset>
  <row>
    <panel>
      <title>Unique Instance Restarts</title>
      <single>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
|  stats dc(host)</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="colorMode">block</option>
        <option name="drilldown">none</option>
        <option name="rangeColors">["0x6db7c6","0x6db7c6"]</option>
        <option name="rangeValues">[0]</option>
        <option name="refresh.display">progressbar</option>
        <option name="trellis.enabled">0</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">medium</option>
        <option name="useColors">1</option>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <title>Host Restart Timeline</title>
      <chart>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
| timechart limit=100 count by host</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">collapsed</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.axisX.scale">linear</option>
        <option name="charting.axisY.scale">linear</option>
        <option name="charting.axisY2.enabled">0</option>
        <option name="charting.axisY2.scale">inherit</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.bubbleMaximumSize">50</option>
        <option name="charting.chart.bubbleMinimumSize">10</option>
        <option name="charting.chart.bubbleSizeBy">area</option>
        <option name="charting.chart.nullValueMode">gaps</option>
        <option name="charting.chart.showDataLabels">none</option>
        <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.chart.style">shiny</option>
        <option name="charting.drilldown">none</option>
        <option name="charting.layout.splitSeries">0</option>
        <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
        <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
        <option name="charting.legend.placement">bottom</option>
        <option name="refresh.display">progressbar</option>
        <option name="trellis.enabled">0</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">medium</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <title>Events</title>
      <table>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
| bucket _time span=5m 
| table _time host _raw 
| sort -_time 
| transaction _time 
| table _time host _raw</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="count">20</option>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</form>

_smp_
Builder
Nice. Thanks!
0 Karma

PowerPacked
Builder

Hi @mbasharat

Please use these searches

index=_audit action=restart_splunkd | stats c by user
index=_audit action=splunkStarting
index=_audit action=splunkShuttingDown

for UI restart its writes in Splunkd_stdout.log

index=_internal sourcetype=splunkd_stdout

Thanks

nick405060
Motivator

Thanks, these queries all work

except for

splunkShuttingDown

which is not a thing, at least in 7.2.0

0 Karma

mbasharat
Builder

THANK YOU!!

0 Karma

adonio
Ultra Champion

hello there,

check the _internal index for "splunkd started" or "(build"
many answers here about that, here are couple examples:
https://answers.splunk.com/answers/242618/how-to-count-the-number-of-times-splunk-is-restart.html
https://answers.splunk.com/answers/105128/how-to-determine-how-long-splunk-has-been-up.html

hope it helps

0 Karma

mbasharat
Builder

Thanks Adonio, is there a way to populate them in a clean list from events. I see my own name in that event and I never started it!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...