Splunk Search

Timechart Command to calculate totals for 7 days

gnovak
Builder

I have a search that I'm using to populate some charts in a dashboard. The search is checking a log and charting the top 20 users who have used the most disk space in their home directories. It works great for a 24 hour period...However I can't seem to get it to work for longer periods such as 7 days or more.

What I'd like to do is that if you choose 7 days, it would display the top 20 users who were using the most space after pretty much calculating space usage for 7 days per user. You'd be able to see the user's space usage over a 7 day period. This would allow you to see when a spike may have occurred that caused a disk issue.

The search I have right now is :

host="ynfs1" sourcetype=userdiskusage earliest=-1d@d latest=-0d@d | rex field=_raw "(?<Space>[\d]+)\s*\/home\/(?<UserName>\S+)" max_match=1000 | search NOT UserName="shares" | table UserName Space | sort -Space | head 20

This search is used to create an area chart on a dashboard. I have a time range picker to change the time from 24 hours to 7 days, but when I do the results are not what I would like. Nothing happens for the most part. Here's my dashboard code:

    <view template="dashboard.html">
  <label>YNFS1 Space Usage</label>
  <module name="AccountBar" layoutPanel="navigationHeader"/>
  <module name="AppBar" layoutPanel="navigationHeader"/>

  <module name="Message" layoutPanel="messaging">
    <param name="filter">*</param>
    <param name="clearOnJobDispatch">False</param>
    <param name="maxSize">1</param>
  </module>

  <module name="TimeRangePicker" layoutPanel="panel_row1_col1">
    <param name="searchWhenChanged">True</param>
    <param name="default">Last 24 hours</param>
      <module name="HiddenSavedSearch" layoutPanel="panel_row2_col1" group="Space Usage by user over a given time" autoRun="True">
        <param name="savedSearch">Andrew ynfs1 search</param>
        <module name="HiddenChartFormatter">
          <param name="charting.chart">area</param>
          <param name="chart.stackMode">stacked</param>
          <param name="secondaryAxisTitle.text">Space</param>
          <module name="JobProgressIndicator"></module>
            <module name="FlashChart">
              <param name="width">100%</param>
              <param name="height">400px</param>
            </module>
          </module>
          <module name="ViewRedirectorLink">
            <param name="viewTarget">flashtimeline</param>
            <param name="popup">True</param>      
          </module> 
      </module>
      <module name="HiddenSavedSearch" layoutPanel="panel_row3_col1" group="Space Usage by user over a given time" autoRun="True">
        <param name="savedSearch">Andrew ynfs1 search</param>
        <module name="HiddenChartFormatter">
          <param name="charting.chart">pie</param>
          <module name="JobProgressIndicator">
            <module name="FlashChart">
              <param name="width">100%</param>
              <param name="height">400px</param>
             </module>
          </module>
          <module name="ViewRedirectorLink">
            <param name="viewTarget">flashtimeline</param>
            <param name="popup">True</param>      
          </module> 
        </module>
     </module>
     <module name="HiddenSavedSearch" layoutPanel="panel_row3_col2" group="Space Usage by user over a given time" autoRun="True">
       <param name="savedSearch">Andrew ynfs1 search</param>
       <module name="SimpleResultsTable"></module>
       <module name="ViewRedirectorLink">
         <param name="viewTarget">flashtimeline</param>
         <param name="popup">True</param>      
       </module> 
    </module>
  </module> 
</view>

I've been looking into timechart as I believe I have to use this command to make this happen. I also tried doing a timechart sum of Space by Username but the results didn't look right. Any ideas?

Tags (1)
0 Karma

RicoSuave
Builder
host="ynfs1" sourcetype=userdiskusage earliest=-1d@d latest=-0d@d | rex field=_raw "(?<Space>[\d]+)\s*\/home\/(?<UserName>\S+)" max_match=1000 | search NOT UserName="shares" | chart sum(Space) over UserName

Does that produce anything when typed in exactly?

0 Karma

gnovak
Builder

at this point i'm thinking the graph should have the time at the bottom, the space on the left (which i might convert to mb eventually) and for the legend have the top 10 or 20 users using the most disk space over time identified by colors

0 Karma

gnovak
Builder

I'm looking at the advanced_intro2 in the UI examples and it's got pretty much the same thing. 😕

0 Karma

gnovak
Builder

well they want the top 20 based on their space consumption. So i said to sort the list and then show me the first 20 in the list.

0 Karma

RicoSuave
Builder

For my use case i don't use the sort command, my users prefer to sort the results themselves by clicking the sort buttons under the day columns.

0 Karma

gnovak
Builder

however this is not showing me the top 20 in the list. host="ynfs1" sourcetype=userdiskusage earliest=-1d@d latest=-0d@d | rex field=_raw "(?[\d]+)\s*\/home\/(?\S+)" max_match=1000 | search NOT UserName="shares" | bucket _time span=1d as day | eval day=strftime(day,"%Y-%m-%d") | chart sum(Space) over UserName | sort -Space | head 20

0 Karma

gnovak
Builder

this produces results host="ynfs1" sourcetype=userdiskusage earliest=-1d@d latest=-0d@d | rex field=_raw "(?[\d]+)\s*\/home\/(?\S+)" max_match=1000 | search NOT UserName="shares" | bucket _time span=1d as day | eval day=strftime(day,"%Y-%m-%d") | chart sum(Space) over UserName | sort -Space

0 Karma

gnovak
Builder

let me try something

0 Karma

gnovak
Builder

yep i got stuff

0 Karma

RicoSuave
Builder

I think the first issue here is that you hardcoded the timerange into the search. try removing the earliest and latest. That should fix your timerange dropdown issue. Try the following search

host="ynfs1" sourcetype=userdiskusage | rex field=_raw "(?[\d]+)\s*\/home\/(?\S+)" max_match=1000 | search NOT UserName="shares" | bucket _time span=1d | eval day=strftime(day,"%Y-%m-%d") | chart sum(Space) over day by UserName

0 Karma

gnovak
Builder

looks like this using the original search i posted.

0 Karma

gnovak
Builder

UserName Space
1 bruce 43236824
2 fhu 41706896
3 lfan 39659892
4 hzhou 32397800
5 gsp 29841020
6 lyuan 29492256
7 mkarimi 27565896
8 jfu 25898072
9 jpfletch 25440672
10 mselvi 25278628

0 Karma

RicoSuave
Builder

what is a typical result for your space field look like?

0 Karma

gnovak
Builder

I'm going to look at the Time Based Charting Tools in UI examples for some hints. ugh!

0 Karma

gnovak
Builder

even this doesn't work. As soon as I put the | chart command nothing happens. host="ynfs1" sourcetype=userdiskusage | rex field=_raw "(?[d]+)s*/home/(?S+)" max_match=1000 | search NOT UserName="shares" | bucket _time span=1d as day | eval day=strftime(day,"%Y-%m-%d") | head 20 | chart sum(Space) by UserName

0 Karma

gnovak
Builder

Also what about the | head 20? How is this search going to show me the top 20 disk hogs over a period of time?

0 Karma

gnovak
Builder

it's not a case issue. that didn't work

0 Karma

gnovak
Builder

also i left the hardcoded time range in the search just to see if the charts did work. my bad

0 Karma

gnovak
Builder

The search stops producing any results as soon as I do the chart command. I tried using timechart but didn't seem to get it right. Also the only results I have is a list of _time and that's it.

0 Karma

RicoSuave
Builder

ah it looks like your rex field extraction is all lowercase. try this:

host="ynfs1" sourcetype=userdiskusage | rex field=_raw "(?[d]+)s*/home/(?S+)" max_match=1000 | search NOT UserName="shares" | bucket _time span=1d as day | eval day=strftime(day,"%Y-%m-%d") | chart sum(space) over day by userame

strftime: This function takes an epochtime value, X, as the first argument and renders it as a string using the format specified by Y.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...