Dashboards & Visualizations

Ever wonder which dashboards are being used and what users are using them?

MattZerfas
Communicator

The dashboard below should help answer that question for you.

The User dropdown uses a |rest search to get a list of LDAP users so if you don't have access to run | rest or aren't using LDAP then that dropdown won't populate but the dashboard will still work fine you just won't be able to look at all dashboard usage for a single user.

You can drilldown on any dashboard that shows in the chart to see the specific users that are using the dashboard per day. To go back to the main chart select the "Reset Drilldown" button.

NOTE - You will need the tokenlinks.js available for the reset button to work. I got it from the 6.x Dashboard Examples App.

 <form script="tokenlinks.js">
    <label>Dashboard Usage</label>
    <fieldset submitButton="true">
      <input type="time" token="field1">
        <label>Max is 30 days back</label>
        <default>
          <earliest>-3d@d</earliest>
          <latest>now</latest>
        </default>
      </input>
      <input type="multiselect" token="user">
        <label>User</label>
        <choice value="*">All Users</choice>
        <search>
          <query>|rest /services/authentication/users splunk_server=local 
   |fields title type realname|rename title as userName|rename realname as Name | search type=LDAP | eval display=userName+" - "+Name | fields userName display</query>
        </search>
        <fieldForLabel>display</fieldForLabel>
        <fieldForValue>userName</fieldForValue>
        <default>*</default>
        <prefix>user=</prefix>
        <delimiter> OR user=</delimiter>
      </input>
    </fieldset>
    <search id="baseDashboardUse">
        <query>index="_internal" user!="-"  sourcetype=splunkd_ui_access "en-US/app"  | rex field=referer "en-US/app/(?<app>[^/]+)/(?<dashboard>[^?/\s]+)" | search  dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alerts" dashboard!="dashboards" dashboard!="reports" dashboard!="report" | bucket _time span=1d | stats dc(dashboard) as c by dashboard user _time  </query>
        <earliest>$field1.earliest$</earliest>
        <latest>$field1.latest$</latest>
     </search>
    <row>
      <panel depends="$field1.earliest$" rejects="$dashboard$">
        <title>Distinct count of users that visited each dashboard per day - (Top 25)</title>
        <chart>
          <title>Select a dashboard to see more info about it</title>
          <search base="baseDashboardUse">
            <query>search $user$ | timechart span=1d limit=25 useother=f count by dashboard</query>           
          </search>
          <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
          <option name="charting.axisLabelsX.majorLabelStyle.rotation">90</option>
          <option name="charting.axisTitleX.visibility">collapsed</option>
          <option name="charting.axisTitleY.visibility">visible</option>
          <option name="charting.axisTitleY2.visibility">visible</option>
          <option name="charting.axisX.scale">linear</option>
          <option name="charting.axisY.scale">linear</option>
          <option name="charting.axisY2.enabled">0</option>
          <option name="charting.axisY2.scale">inherit</option>
          <option name="charting.chart">column</option>
          <option name="charting.chart.bubbleMaximumSize">50</option>
          <option name="charting.chart.bubbleMinimumSize">10</option>
          <option name="charting.chart.bubbleSizeBy">area</option>
          <option name="charting.chart.nullValueMode">gaps</option>
          <option name="charting.chart.showDataLabels">none</option>
          <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
          <option name="charting.chart.stackMode">stacked</option>
          <option name="charting.chart.style">shiny</option>
          <option name="charting.drilldown">all</option>
          <drilldown>
            <set token="dashboard">$click.name2$</set>
          </drilldown>
          <option name="charting.layout.splitSeries">0</option>
          <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
          <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
          <option name="charting.legend.placement">right</option>
          <option name="height">800</option>
        </chart>
      </panel>
    </row>
    <row>
      <panel depends="$dashboard$">
        <title>Distinct count of users that visited each dashboard per day</title>
        <chart>
          <search base="baseDashboardUse">
            <query>search dashboard=$dashboard$ | timechart span=1d limit=25 useother=f count by dashboard</query>           
          </search>
          <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
          <option name="charting.axisLabelsX.majorLabelStyle.rotation">90</option>
          <option name="charting.axisTitleX.visibility">collapsed</option>
          <option name="charting.axisTitleY.visibility">visible</option>
          <option name="charting.axisTitleY2.visibility">visible</option>
          <option name="charting.axisX.scale">linear</option>
          <option name="charting.axisY.scale">linear</option>
          <option name="charting.axisY2.enabled">0</option>
          <option name="charting.axisY2.scale">inherit</option>
          <option name="charting.chart">column</option>
          <option name="charting.chart.bubbleMaximumSize">50</option>
          <option name="charting.chart.bubbleMinimumSize">10</option>
          <option name="charting.chart.bubbleSizeBy">area</option>
          <option name="charting.chart.nullValueMode">gaps</option>
          <option name="charting.chart.showDataLabels">none</option>
          <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
          <option name="charting.chart.stackMode">stacked</option>
          <option name="charting.chart.style">shiny</option>
          <option name="charting.drilldown">none</option>
          <option name="charting.layout.splitSeries">0</option>
          <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
          <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
          <option name="charting.legend.placement">bottom</option>
          <option name="height">400</option>
        </chart>
      </panel>
    </row>
    <row>
      <panel>
        <title>Distinct users that visited $dashboard$</title>
        <html depends="$dashboard$">
           <button class="btn" data-unset-token="dashboard">Reset Drilldown</button>
        </html>
        <table depends="$dashboard$">
         <search base="baseDashboardUse">
            <query>search dashboard=$dashboard$ | stats values(user) as "Unique Users" by _time</query>           
         </search>
          <option name="wrap">true</option>
          <option name="rowNumbers">false</option>
          <option name="dataOverlayMode">none</option>
          <option name="drilldown">cell</option>
          <option name="count">10</option>
        </table>
      </panel>
    </row>
  </form>

alt text
alt text

1 Solution

MattZerfas
Communicator

If this helped you upvote this to show that it would be nice to have more dashboards like this out of the box for admins to see what users are doing. If you used this and made any enhancements/changes please comment below so all of us can benefit from them.

Thanks!

View solution in original post

manishmittal12
Explorer

@somesoni2 - thanks for the query. i will check on Monday if i have access to rest api and see how that search goes.

0 Karma

manishmittal12
Explorer

@somesoni2 - i am not after historical data. i am just trying to do some cleanup activity as the number of dashboard have grown to more than 300.
So i want to find out which dashboard are not opened at all i.e not used in last 30 days.

your query will give usage statistics right.

0 Karma

somesoni2
Revered Legend

Ohhh, I guess I misread your question.

Yes, this query tell which dashboards were actually being used in selected time range. For the thing that you're looking for, you need to basically find list of all dashboards and then find diffs between all dashboard list and dashboard list from this query, which should give you which dashboards were NOT used. Something like this

index="_internal" source=/opt/splunk/var/log/splunk/splunkd_ui_access.log  "en-US/app" | rex "(?i)^[^\-]*\-\s+(?P<user>[^ ]+)" | rex "en-US/app/(?<app>[^/]+)/(?<dashboard>[^?/\"\s]+)"| search app="search" dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alert*" dashboard!="dashboards" dashboard!="reports" dashboard!="report" dashboard!="@go" dashboard!="%40go" dashboard!="field_extractor" dashboard!="pivot" user!="-" 
| stats count by app dashboard | table app dashboard | eval isUsed=1
| append [| rest /servicesNS/-/-/data/ui/views splunk_server=local | table eai:acl.app title | rename eai:acl.app as app title as dashboard | search app="search" dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alert*" dashboard!="dashboards" dashboard!="reports" dashboard!="report" dashboard!="@go" dashboard!="%40go" dashboard!="field_extractor" dashboard!="pivot" | eval isUsed=0]
| stats max(isUsed) as isUsed by app dashboard | where isUsed=0

manishmittal12
Explorer

@somesoni2 - do you have any similar search query which tells dashboards not being used since more than 30 days.

0 Karma

somesoni2
Revered Legend

That's the only query. It uses index="_internal", so how far back you can look for depends upon the retention period of index="_internal". Default is 30 days, so that's what you get. You can increase it's retention period (or setup a summary index search to periodically save the data) in case you wanted more historical data.

0 Karma

bcyates
Communicator

I'm a little late to this party but I do a copy and paste and it errors out with 'Encountered the following error while trying to update: In handler 'views': Error parsing XML on line 120: Premature end of data in tag form line 1'

Anyone know what I am missing?

0 Karma

MattZerfas
Communicator

If you are doing a direct copy/paste to the source of the dashboard you will need to do a find and replace for any < or > 's with &lt; or &gt; 's in the panel search queries.

sstumpf_splunk
Splunk Employee
Splunk Employee

Believe this app will now do what you need and more, but it's not free.

https://splunkbase.splunk.com/app/3541/

0 Karma

machiel
Path Finder

DO NOT COMPLETELY RELY ON CURRENT USAGE STATISTICS USING THE ANSWER OF THIS QUESTION
Please note that the solution of finding dashboard usage using the sourcetype=splunkd_ui_access, is not very reliable.

What about the case when you have views having the same name, but in different apps?
Example situation: you have App_welcome, App1 and App2.
App1 and App2 contain both the views "DashboardA" and "DashboardB".

It is possible to refer to a view not existing in an app, let's define the navigation in App_welcome (which has the navigation to all views that should be visible):

<nav search_view="search" color="#65A637">
  <view name="DashboardA" default="true" />
</nav>

The question is now to which DashboardA does the navigation point to? The answer from my local Splunk instance: App2 (because Splunk somehow selects the last app in a list of alphabetically ordered apps that contain a 'DashboardA')

The usage statistics issue at hand
When a user clicks the link to DashboardA, you get an event in the splunkd_ui_access sourcetype that tells you the user visited "http://localhost:8000/en-US/app/App_welcome/DashboardA" !!!
So the issue is that you must have views with unique names in your whole environment, since the app-part of the event in the splunkd_ui_access is unreliable. It does not tell you which DashboardA was viewed, furthermore, from the event you might erroneously conclude that DashboardA is located in the App_welcome!

Probably the best solution for getting reliable usage statistics is to wait for Splunk to come up with a well-designed and fully supported solution to present insight in dashboard usage statistics.

MattZerfas
Communicator

I see what you are getting at @machiel and I agree with you but until Splunk releases an official solution this works for most. My instance actually only uses 1 app(search) for all dashboards so this solution works 100% for us.

0 Karma

somesoni2
Revered Legend

Here is the same dashboard with following updates

1) Replace all source references using splunkd_ui_acc.log references to just "sourcetype=splunkd_ui_access", as it'll always work regardless of OS, install directory etc. changes.
2) Remove app=search filter as there can be dashboard in different app
3) Used post-process for performance improvement.

<form script="tokenlinks.js">
   <label>Dashboard Usage</label>
   <fieldset submitButton="true">
     <input type="time" token="field1">
       <label>Max is 30 days back</label>
       <default>
         <earliest>-3d@d</earliest>
         <latest>now</latest>
       </default>
     </input>
     <input type="multiselect" token="user">
       <label>User</label>
       <choice value="*">All Users</choice>
       <search>
         <query>|rest /services/authentication/users splunk_server=local 
  |fields title type realname|rename title as userName|rename realname as Name | search type=LDAP | eval display=userName+" - "+Name | fields userName display</query>
       </search>
       <fieldForLabel>display</fieldForLabel>
       <fieldForValue>userName</fieldForValue>
       <default>*</default>
       <prefix>user=</prefix>
       <delimiter> OR user=</delimiter>
     </input>
   </fieldset>
   <search id="baseDashboardUse">
       <query>index="_internal" user!="-"  sourcetype=splunkd_ui_access "en-US/app"  | rex field=referer "en-US/app/(?<app>[^/]+)/(?<dashboard>[^?/\s]+)" | search  dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alerts" dashboard!="dashboards" dashboard!="reports" dashboard!="report" | bucket _time span=1d | stats dc(dashboard) as c by dashboard user _time  </query>
       <earliest>$field1.earliest$</earliest>
       <latest>$field1.latest$</latest>
    </search>
   <row>
     <panel depends="$field1.earliest$" rejects="$dashboard$">
       <title>Distinct count of users that visited each dashboard per day - (Top 25)</title>
       <chart>
         <title>Select a dashboard to see more info about it</title>
         <search base="baseDashboardUse">
           <query>search $user$ | timechart span=1d limit=25 useother=f count by dashboard</query>           
         </search>
         <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
         <option name="charting.axisLabelsX.majorLabelStyle.rotation">90</option>
         <option name="charting.axisTitleX.visibility">collapsed</option>
         <option name="charting.axisTitleY.visibility">visible</option>
         <option name="charting.axisTitleY2.visibility">visible</option>
         <option name="charting.axisX.scale">linear</option>
         <option name="charting.axisY.scale">linear</option>
         <option name="charting.axisY2.enabled">0</option>
         <option name="charting.axisY2.scale">inherit</option>
         <option name="charting.chart">column</option>
         <option name="charting.chart.bubbleMaximumSize">50</option>
         <option name="charting.chart.bubbleMinimumSize">10</option>
         <option name="charting.chart.bubbleSizeBy">area</option>
         <option name="charting.chart.nullValueMode">gaps</option>
         <option name="charting.chart.showDataLabels">none</option>
         <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
         <option name="charting.chart.stackMode">stacked</option>
         <option name="charting.chart.style">shiny</option>
         <option name="charting.drilldown">all</option>
         <drilldown>
           <set token="dashboard">$click.name2$</set>
         </drilldown>
         <option name="charting.layout.splitSeries">0</option>
         <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
         <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
         <option name="charting.legend.placement">right</option>
         <option name="height">800</option>
       </chart>
     </panel>
   </row>
   <row>
     <panel depends="$dashboard$">
       <title>Distinct count of users that visited each dashboard per day</title>
       <chart>
         <search base="baseDashboardUse">
           <query>search dashboard=$dashboard$ | timechart span=1d limit=25 useother=f count by dashboard</query>           
         </search>
         <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
         <option name="charting.axisLabelsX.majorLabelStyle.rotation">90</option>
         <option name="charting.axisTitleX.visibility">collapsed</option>
         <option name="charting.axisTitleY.visibility">visible</option>
         <option name="charting.axisTitleY2.visibility">visible</option>
         <option name="charting.axisX.scale">linear</option>
         <option name="charting.axisY.scale">linear</option>
         <option name="charting.axisY2.enabled">0</option>
         <option name="charting.axisY2.scale">inherit</option>
         <option name="charting.chart">column</option>
         <option name="charting.chart.bubbleMaximumSize">50</option>
         <option name="charting.chart.bubbleMinimumSize">10</option>
         <option name="charting.chart.bubbleSizeBy">area</option>
         <option name="charting.chart.nullValueMode">gaps</option>
         <option name="charting.chart.showDataLabels">none</option>
         <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
         <option name="charting.chart.stackMode">stacked</option>
         <option name="charting.chart.style">shiny</option>
         <option name="charting.drilldown">none</option>
         <option name="charting.layout.splitSeries">0</option>
         <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
         <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
         <option name="charting.legend.placement">bottom</option>
         <option name="height">400</option>
       </chart>
     </panel>
   </row>
   <row>
     <panel>
       <title>Distinct users that visited $dashboard$</title>
       <html depends="$dashboard$">
          <button class="btn" data-unset-token="dashboard">Reset Drilldown</button>
       </html>
       <table depends="$dashboard$">
        <search base="baseDashboardUse">
           <query>search dashboard=$dashboard$ | stats values(user) as "Unique Users" by _time</query>           
        </search>
         <option name="wrap">true</option>
         <option name="rowNumbers">false</option>
         <option name="dataOverlayMode">none</option>
         <option name="drilldown">cell</option>
         <option name="count">10</option>
       </table>
     </panel>
   </row>
 </form>

MattZerfas
Communicator

Somesoni2 I have updated my original post with your fixes/changes if you want to remove this one. Thanks for the extra improvements!!!

vinceskahan
Path Finder

A screenshot would help us know to bother (or not).....

0 Karma

MattZerfas
Communicator

Vince I have added 2 screenshots to the original post.

0 Karma

dbcase
Motivator

Odd, I see the dropdowns but I get Search returned no results..... We aren't using LDAP and I'm admin. I did a straight copy/paste.

0 Karma

Jewatson17
Path Finder

hmm not working for me. Keep getting parsing error saying "Unclosed root tag, but I checked and is at the end. Any help with this please? Does this still work?

0 Karma

somesoni2
Revered Legend

If you're not using LDAP authentication then update the multiselect query to change | search type=LDAP with | search type=Splunk

0 Karma

dbcase
Motivator

will give that a try.... I hacked at the first query and came up with this

index="_internal"  sourcetype=splunkd_ui_access   |rex field=referer "en-US/app/(?<app>[^/]+)/(?<dashboard>[^?/\s]+)" `|search app="search" dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alerts" dashboard!="dashboards" dashboard!="reports" dashboard!="report" |`bucket _time span=1d | stats dc(dashboard) as c by dashboard user _time  | timechart span=1d limit=25 useother=f count by dashboard

but it fails at the section below

 |search app="search" dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alerts" dashboard!="dashboards" dashboard!="reports" dashboard!="report" |
0 Karma

dbcase
Motivator

using | search type=Splunk got the users to populate, thanks! Dashboard is still returning zero results when ALL users is selected. If I select a single user (me) I get the same blank dashboard result.

0 Karma

dbcase
Motivator

this query doesn't seem to work for me

<query>index="_internal" source=*access* user!="-" $user$ source="/opt/splunk/var/log/splunk/splunkd_ui_access.log" "en-US/app"  | rex field=referer "en-US/app/(?&lt;app&gt;[^/]+)/(?&lt;dashboard&gt;[^?/\s]+)" | search app="search" dashboard!="job_management" dashboard!="dbinfo" dashboard!="*en-US" dashboard!="search" dashboard!="home" dashboard!="alerts" dashboard!="dashboards" dashboard!="reports" dashboard!="report" | bucket _time span=1d | stats dc(dashboard) as c by dashboard user _time  | timechart span=1d limit=25 useother=f count by dashboard</query>

even with the modification

Replace all source references using splunkd_ui_acc.log references to just "`sourcetype=splunkd_ui_access`"
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.