Splunk Search

Has anyone else got inconsistent results in Splunk?

joe06031990
Communicator

Hi,

on our Splunk instance I have set a report using a time chart with a span of 1h and time frame of a day and the report is scheduled to run every hour however each time the report runs it shows different results. Just wondered if anyone has seen this before?

 

thanks,

 

joe

Labels (4)
0 Karma
1 Solution

joe06031990
Communicator

Looks like there was a fault with two of the search head nodes.

View solution in original post

0 Karma

bowesmana
SplunkTrust
SplunkTrust

What is your time range set to, i.e. what is the exact earliest/latest in the search definition.

If you have your 'end' time as now, then it will search up to now, so naturally each hour will have different results.

When you say time frame of a day, do you mean 24h.

Can you expand on what you mean by 'different results'. In what way?

 

0 Karma

joe06031990
Communicator

Hi ,

the timeframe is set to today and the span in the time chart is 1 hour.

sometime the volume is lower or higher from the same hour.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

So, at 10am it runs it gives 10 values for the first 10 hours and at 11 am you have 11 values, and are you saying that ANY of the first 10 can have different values or just the value for 10am?

What is the 'ending' time of the search in 'Today'? Is it now or @h 

If it's now, it will be somewhat vague, as it may not contain events that are being indexed at that time, or events that maybe arrive one or two minutes after the search has run, but which have slightly earlier times.

One way to see if you have event 'lag' is to look at _indextime field to see how much difference there is between that and _time. 

If _time is some time before _indextime, you have lag

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

In what way are they different?

Have a look at the job inspector to see how many events are processed at each stage.

0 Karma

joe06031990
Communicator

Looks like there was a fault with two of the search head nodes.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...