All Apps and Add-ons

How to find if the Splunk events are in future?

Splunk Employee
Splunk Employee

Is there easy way to find if the data is in future. We have ingested data into splunk, but don't see them in dashboard. We are looking for easy way to identify such data.

SplunkTrust
SplunkTrust

As per the comment from skoelpin you can either use the advanced settings or in your search add:

earliest=+5m latest=+10y

Or similar...

In the app Alerts For Splunk Admins I have an alert called "IndexerLevel - Future Dated Events that appeared in the last week", for this exact purpose.
If you want just the one search it's in github here

SplunkTrust
SplunkTrust

You can simply select the timerange picker and select Advanced.. Earliest=+1s Latest=+2mon

Or if you wanted to do it all in SPL, you could do this

index=... earliest=+1s latest=+2mon
0 Karma

Splunk Employee
Splunk Employee

Here is a test I did. Created a sample test.log file with data for 2019

64.242.88.10 - - [07/Mar/2019:16:58:54 -0800] "GET /mailman/listinfo/administration HTTP/1.1" 200 6459
lordgun.org - - [07/Mar/2019:17:01:53 -0800] "GET /razor.html HTTP/1.1" 200 2869
64.242.88.10 - - [07/Mar/2019:17:09:01 -0800] "GET /twiki/bin/search/Main/SearchResult?scope=text®ex=on&search=Joris%20*Benschop[^A-Za-z] HTTP/1.1" 200 4284
64.242.88.10 - - [07/Mar/2019:17:10:20 -0800] "GET /twiki/bin/oops/TWiki/TextFormattingRules?template=oopsmore¶m1=1.37¶m2=1.37 HTTP/1.1" 200 11400
64.242.88.10 - - [07/Mar/2019:17:13:50 -0800] "GET /twiki/bin/edit/TWiki/DefaultPlugin?t=1078688936 HTTP/1.1" 401 12846
64.242.88.10 - - [07/Mar/2019:17:16:00 -0800] "GET /twiki/bin/search/Main/?scope=topic®ex=on&search=^g HTTP/1.1" 200 3675
64.242.88.10 - - [07/Mar/2019:17:17:27 -0800] "GET /twiki/bin/search/TWiki/?scope=topic®ex=on&search=^d HTTP/1.1" 200 5773
lj1036.inktomisearch.com - - [07/Mar/2019:17:18:36 -0800] "GET /robots.txt HTTP/1.0" 200 68
lj1090.inktomisearch.com - - [07/Mar/2019:17:18:41 -0800] "GET /twiki/bin/view/Main/LondonOffice HTTP/1.0" 200 3860
64.242.88.10 - - [07/Mar/2019:17:21:44 -0800] "GET /twiki/bin/attach/TWiki/TablePlugin HTTP/1.1" 401 12846
64.242.88.10 - - [07/Mar/2019:17:22:49 -0800] "GET /twiki/bin/view/TWiki/ManagingWebs?rev=1.22 HTTP/1.1" 200 9310
64.242.88.10 - - [07/Mar/2019:17:23:54 -0800] "GET /twiki/bin/statistics/Main HTTP/1.1" 200 808
64.242.88.10 - - [07/Mar/2019:17:26:30 -0800] "GET /twiki/bin/view/TWiki/WikiCulture HTTP/1.1" 200 5935 

Today is 03/29/2018 17:26:11

After that data is indexed we need to check if the data is in future.

source="*test.log.log" host="test" sourcetype="apache" | eval delay=_indextime-_time | convert ctime(_time) AS xtime |convert ctime(_indextime) AS indextime | table _raw delay _time indextime xtime

It shows that data is early by 773981.

We can see same information in metrics.log

03-29-2018 17:26:12.099 -0700 INFO Metrics - group=per_sourcetype_thruput, series="test", kbps=0.1403387022431233, eps=1.258039598828591, kb=4.3505859375, ev=39, avg_age=776259.9487179487, max_age=778822
03-29-2018 17:26:12.098 -0700 INFO Metrics - group=per_host_thruput, series="test", kbps=0.1403387022431233, eps=1.258039598828591, kb=4.3505859375, ev=39, avg_age=776259.9487179487, max_age=778822