Timechart "yesterday" forced to display full 24 hours

[frankloron]

I have a feeling there is a simple solution to this, I am just not seeing it. Possibly appending null data at the start and end of the time range.

GOAL: I want to create a dashboard showing "Yesterday", and 2 rows. First row contains a chart of multiple usage metrics (CPU, swap, lots of lines), right under that is a row with a bar chart of "events". Both charts should display the exact same time span. If the user picks "Yesterday", both graphs should show Midnight to Midnight even if there are only a few "events" in the second chart.

I do not want to overlay two chart types. I have other Advanced XML charts with overlays, but this set of data is too messy, and the charts need to be easy to read. Advanced XML okay if I can use that to force the chart x-axis range.

Example (EDITED to show full search):
sourcetype="alert_log" | rex field=_raw "(?i)(?:[^ ]* ){6}(?P<alert_type>\S+)\s+\S+\s+(?P<alert_level>\S+)\s+on\s+(?P<alert_host>\S+)\s+\((?P<alert_subcategory>[^\)]+)\):{0,1}(?P<alert_message>.*)$" | search alert_host="$hostname$" |stats count(alert_host) by _time
    returns 
            _time   count(alert_host)
    1   7/25/13 1:15:10.000 AM  1
    2   7/25/13 3:05:05.000 AM  1
    3   7/25/13 3:20:05.000 AM  1
    4   7/25/13 4:00:05.000 AM  1
    5   7/25/13 4:15:05.000 AM  1
    6   7/25/13 4:35:05.000 AM  1
    7   7/25/13 4:45:05.000 AM  2
    8   7/25/13 5:10:05.000 AM  1
    9   7/25/13 7:00:05.000 AM  1
    10  7/25/13 8:20:05.000 AM  1

I have tried the chart two ways, both give the same result, only shows 1 am to 8 am not the full 24 hours.

.. | search alert_host="$hostname$" |timechart count
or 
.. | search alert_host="$hostname$" | timechart fixedrange=true count

We are using Splunk 5.0.1

Edit - Additional Example
Here is the build-up of what I have. First "alert_log" rows look like:
"Sun Jul 28 23:55:04 2013 - appload [15712] OK on hostname508 (P_5555)"
or
"Sun Jul 28 15:50:04 2013 - load [30740] NOTICE on hostname508 (load): load (37.54) is above 10"

When this is run with "Yesterday" timespan for one host and all events, it shows events 7/28/13 12:00 AM through 7/28/13 11:30 PM as expected due to the heartbeat events. Add alert_type!=OK to remove the heartbeats.

sourcetype="alert_log" | rex field=_raw "(?i)(?:[^ ]* ){6}(?P<alert_type>\S+)\s+\S+\s+(?P<alert_level>\S+)\s+on\s+(?P<alert_host>\S+)\s+\((?P<alert_subcategory>[^\)]+)\):{0,1}(?P<alert_message>.*)$" | search alert_host="hostname508" alert_level!=OK |timechart count

Only 4 Events occured at 8:20 AM yesterday. The output timechart has only one column at 8:20 AM, table view has just one row.

Thanks in advance --AFL

[Keysofsandiego]

FYI ... I was also having some trouble with the 24 hour fixed time. I am editing the source code form a dashboard so I hope this helps I chose to go another route and it worked well for me.
I made a timechart that would grab data for a specific 24 hour period (previous day hardcoded by earliest/latest time), and would bucket results in 5 minute buckets, fill the value null with "zero", do a fixed range (this is the command that makes the range fixed in that 24 hour period)... then the rest is just some fun custom stuff. color etc...
You just need to replace the search, earliest/latest time, and max(XXX) to the custom field you are wanting to chart.

  <row>
    <panel>
      <title>TITLE OF CHART</title>
      <chart>
        <searchString>
          YOURSEARCH FOR SERVER/USER/FOO/BAR etc... earliest=3/12/2015:00:00:00 latest=3/13/2015:00:00:00 
            | bucket _time span="5m" | fillnull value=NoVal | timechart fixedrange=t span="5m" max(YOURFIELD) 
        </searchString>
        <option name="rowNumbers">undefined</option>
        <option name="charting.chart">line</option>
        <option name="charting.chart.nullValueMode">gaps</option>
        <option name="charting.chart.stackMode">stacked</option>
        <option name="charting.drilldown">all</option>
        <option name="charting.legend.placement">none</option>
        <option name="charting.seriesColors">[0x000000]</option>
        <option name="charting.drilldown">all</option>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">visible</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.axisX.scale">linear</option>
        <option name="charting.axisY.scale">linear</option>
        <option name="charting.axisY2.enabled">false</option>
        <option name="charting.axisY2.scale">inherit</option>
        <option name="charting.chart.bubbleMaximumSize">50</option>
        <option name="charting.chart.bubbleMinimumSize">10</option>
        <option name="charting.chart.bubbleSizeBy">area</option>
        <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
        <option name="charting.chart.style">shiny</option>
        <option name="charting.layout.splitSeries">0</option>
        <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
      </chart>
    </panel>
  </row>

For the same code just in a New search you can try this and click on visiualizations...

YOUR SERVER/FOO earliest=3/12/2015:00:00:00 latest=3/13/2015:00:00:00 
| YOUR CUSTOM FIELD ...
| bucket _time span="5m" 
| fillnull value=NoVal 
| timechart fixedrange=t span="5m" max(YOURFIELD)

Good luck

[David · Accepted solution]

I'm sure you have resolved this over the past five months, but in case anyone else stumbles across this same question...

I just encountered something very similar to this where I was creating a dashboard that looked at the number of alerts over the previous two months, and could not get the chart to show the entire two months. I tried fixedrange=(false|true), I tried switching to | bucket _time span=1d | stats count and several other things to get it to work, to no avail.

The answer ended up being using fillnull so that my original search:

product="Anti Malware" Confidence_Level>=3 | timechart count by Confidence_Level

Became:

product="Anti Malware" Confidence_Level>=3 | fillnull value=NoVal | timechart count by Confidence_Level

and my chart suddenly began showing the entire period of time.

This doesn't really seem in line with the documentation for timechart or fillnull, but in either case my report now looks correct.

[laserval]

This is great, tried the same thing and it works well!

[frankloron]

David, you got it! I had not solved this over the last 5 months and I appreciate the answer. Adding a fillnull before the timechart did the trick!

[frankloron]

Thanks Sideview! I feel like this should work too. I will try something similar with a different data set and/or use the inspector to see why it is clamping down.

[sideview]

Is this happening just in the normal search interface (aka "flashtimeline"), or is this happening in a custom view? We're missing something here and I think it's something unusual, possibly about how the view is applying the timerange argument. Basically in this situation, timechart should indeed be returning 24 rows. The only time it ever "clamps down" on the data like this is when there is no timerange submitted and the search runs over all time. Or more generally when the search API thinks there's no timerange submitted.

[HiroshiSatoh]

And what you do?
･･････ earliest=-1d@h latest=@h ･･････|timechart count

[HiroshiSatoh]

I answered that I think that you have specified a time range direct time range picker if funny.

It was not a answer. Please forget.

[frankloron]

I do not understand this answer. Can you give a longer example?

[sideview]

Can you post the entire search? There must be something going on in your search language, because if this were a simple streaming search piped into timechart, it would indeed show midnight to midnight even regardless of where the actual data fell. Or better yet post the full XML if you can.