Splunk Search
Highlighted

Why does the timechart command display inconsistent results when the time range is changed?

Path Finder

When I use the following search (some criteria obfuscated for security):

index=main sourcetype=transaction application=foo component=bar  customerCode=x Type=y messageType=z | timechart span=1d count as count

and I set the time range to a single day (9th of jan 2017), the resulting table shows a single result:

_time                         count
2017-01-09T00:00:00.000+0100    1

as I would expect, this is consistent with the event I find when I omit the timechart command entirely, but when I change the time range to the whole month of jan 2017, I get this (just showing first 14 results)

_time                         count
2017-01-01T00:00:00.000+0100    0
2017-01-02T00:00:00.000+0100    0
2017-01-03T00:00:00.000+0100    0
2017-01-04T00:00:00.000+0100    0
2017-01-05T00:00:00.000+0100    3
2017-01-06T00:00:00.000+0100    0
2017-01-07T00:00:00.000+0100    0
2017-01-08T00:00:00.000+0100    0
2017-01-09T00:00:00.000+0100    0
2017-01-10T00:00:00.000+0100    0
2017-01-11T00:00:00.000+0100    0
2017-01-12T00:00:00.000+0100    15
2017-01-13T00:00:00.000+0100    25

suddenly there is nothing counted on the 9th, how can this happen?

in fact, if I click the cell that says zero in the table that gets returned, and select get view events, it takes me to the event that I would expect to be counted here:

2017-01-09T13:36:56.109+0100 TRAN b53a13ca-e1bc-4e64-964c-09c4714ba40e custom-operations process-engine 127.0.1.1 type:y|customerCode:x|duration:1327|bytesAllocated:15692632|executorUtilPct:0.0|messageType:z

some data here is modified for security reasons, but that should not affect the anwswers

Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

SplunkTrust
SplunkTrust

Please post the entire search that got you the last results.

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

Path Finder

It's the same search, but with a different timerange

index=main sourcetype=transaction application=foo component=bar customerCode=x Type=y messageType=z | timechart span=1d count as count

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

SplunkTrust
SplunkTrust

Hmmm. How is the different time range being entered? standard search or dashboard?

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

Path Finder

Standard search, (I discovered it when I had a weird result in a dashboard, but I opened it in search)

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

SplunkTrust
SplunkTrust

Shot in the dark, but simplify "count as count" to "count" -- and verify that there is no existing field called "count" on the events -- and see what happens.

I've noticed that splunk has occasional trouble distinguishing between the count it's doing at any given time and the count that is a field already on an event record.

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

Path Finder

thanks for your suggestion but it makes no difference

index=main sourcetype=transaction application=foo component=bar
customerCode=x Type=y messageType=z | timechart span=1d count

gives the same result,

I'm positive that there is no count field, and
the same query with dc(uid) also gives no result (uid is the b53a13ca-e1bc-4e64-964c-09c4714ba40e code in the event above)

I also tried it with putting a | fields command before the timechart to rule out what you suggest:

index=main sourcetype=transaction application=foo component=bar
customerCode=x Type=y messageType=z | fields uid | timechart span=1d count

and that makes no difference either

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

SplunkTrust
SplunkTrust

Do you get results for jan 9 when you run this??

index=main sourcetype=transaction application=foo component=bar 
customerCode=x Type=y messageType=z | bucket span=1d _time | stats count by _time
0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

Path Finder

tried that and I get this

_time                         count
2017-01-05T00:00:00.000+0100    3
2017-01-12T00:00:00.000+0100    15
2017-01-13T00:00:00.000+0100    25
2017-01-16T00:00:00.000+0100    25
2017-01-19T00:00:00.000+0100    7
2017-01-20T00:00:00.000+0100    13
2017-01-24T00:00:00.000+0100    4

so basically no, no results for jan 9,
this is what I would get if I ran the original timechart command with cont=false (verified)

0 Karma
Highlighted

Re: Why does the timechart command display inconsistent results when the time range is changed?

SplunkTrust
SplunkTrust

Hmmm. I'm seeing nothing at all. There's only one handle left to pull on. Try it without the timespan parameter, or with different timespan parameters - 8h or something.

0 Karma