Dashboards & Visualizations

Splunk Restart Tracking

mbasharat
Contributor

Hi folks,

I want to setup a dashboard to track Splunk activities. I need to know how to track who restarted Splunk via both UI and audit logs in Splunk Dashboard?

Thanks in-advance.

Tags (1)
0 Karma
1 Solution

PowerPacked
Builder

Hi @mbasharat

Please use these searches

index=_audit action=restart_splunkd | stats c by user
index=_audit action=splunkStarting
index=_audit action=splunkShuttingDown

for UI restart its writes in Splunkd_stdout.log

index=_internal sourcetype=splunkd_stdout

Thanks

View solution in original post

bandit
Motivator

Splunk Restart Dashboard

<form theme="dark">
  <label>Splunk Restarts</label>
  <fieldset submitButton="false">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="host_include_pattern" searchWhenChanged="true">
      <label>Host Include Pattern (host1,host2)</label>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="host_exclude_pattern" searchWhenChanged="true">
      <label>Host Exclude Pattern</label>
      <default>null</default>
      <prefix>NOT host="*</prefix>
      <suffix>*"</suffix>
    </input>
  </fieldset>
  <row>
    <panel>
      <title>Unique Instance Restarts</title>
      <single>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
|  stats dc(host)</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="colorMode">block</option>
        <option name="drilldown">none</option>
        <option name="rangeColors">["0x6db7c6","0x6db7c6"]</option>
        <option name="rangeValues">[0]</option>
        <option name="refresh.display">progressbar</option>
        <option name="trellis.enabled">0</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">medium</option>
        <option name="useColors">1</option>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <title>Host Restart Timeline</title>
      <chart>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
| timechart limit=100 count by host</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">collapsed</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.axisX.scale">linear</option>
        <option name="charting.axisY.scale">linear</option>
        <option name="charting.axisY2.enabled">0</option>
        <option name="charting.axisY2.scale">inherit</option>
        <option name="charting.chart">column</option>
        <option name="charting.chart.bubbleMaximumSize">50</option>
        <option name="charting.chart.bubbleMinimumSize">10</option>
        <option name="charting.chart.bubbleSizeBy">area</option>
        <option name="charting.chart.nullValueMode">gaps</option>
        <option name="charting.chart.showDataLabels">none</option>
        <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.chart.style">shiny</option>
        <option name="charting.drilldown">none</option>
        <option name="charting.layout.splitSeries">0</option>
        <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
        <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
        <option name="charting.legend.placement">bottom</option>
        <option name="refresh.display">progressbar</option>
        <option name="trellis.enabled">0</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">medium</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <title>Events</title>
      <table>
        <search>
          <query>index=_internal sourcetype=splunkd source="*splunkd.log" "Splunkd starting" host IN ($host_include_pattern$) $host_exclude_pattern$ 
| bucket _time span=5m 
| table _time host _raw 
| sort -_time 
| transaction _time 
| table _time host _raw</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
          <refresh>30s</refresh>
          <refreshType>delay</refreshType>
        </search>
        <option name="count">20</option>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</form>

_smp_
Builder
Nice. Thanks!
0 Karma

PowerPacked
Builder

Hi @mbasharat

Please use these searches

index=_audit action=restart_splunkd | stats c by user
index=_audit action=splunkStarting
index=_audit action=splunkShuttingDown

for UI restart its writes in Splunkd_stdout.log

index=_internal sourcetype=splunkd_stdout

Thanks

nick405060
Motivator

Thanks, these queries all work

except for

splunkShuttingDown

which is not a thing, at least in 7.2.0

0 Karma

mbasharat
Contributor

THANK YOU!!

0 Karma

adonio
Ultra Champion

hello there,

check the _internal index for "splunkd started" or "(build"
many answers here about that, here are couple examples:
https://answers.splunk.com/answers/242618/how-to-count-the-number-of-times-splunk-is-restart.html
https://answers.splunk.com/answers/105128/how-to-determine-how-long-splunk-has-been-up.html

hope it helps

0 Karma

mbasharat
Contributor

Thanks Adonio, is there a way to populate them in a clean list from events. I see my own name in that event and I never started it!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...