Alltime-Realtime Able to See Data - Zero Data for Historical Searches

[bitofrumncoke]

Strangest thing. I have some Infoblox logs coming in from a Syslog-NG server where we have a UF installed. UF is successfully sending the Infoblox logs to Splunk BUT, I can only see those logs when doing an alltime-realtime search but can't see them anywhere when doing a historical alltime search even when logged in as admin. I can search other logs in the same index but just comes back with "0 events" and no errors in the job - just nothing. Can't find them via sourcetype, source or host.

Any ideas? I know the data is there but just can't see it on historical searches. 

[bitofrumncoke]

Thanks for the response! Logs are in UTC time it seems so a bit in the future but all time should show data anyway. Still, ran another search for 1 year in the future and 1 year in the past at the same time - still zero data returned with no errors. 

[Yorokobi]

Is the date/time in those syslog events far into the future or past? If they're in the future, you can try searching with earliest=now latest=+5y (for example). If they're too far into the past, Splunk is probably dropping them. Both of these scenarios are logged in the indexers' _internal index.