Splunk Enterprise

Why is there a delay in index time and search time data?

dntest
New Member

Delay in index time and search time data..
There is a delay of 10 hours 

index=test_shift  "*10987867*" | eval indextime=strftime(_indextime,"%d/%m/%Y %H:%M:%S")
| table _raw _time indextime



_time is 2022-11-15 13:42:31
indextime is 2022-11-15 23:27:33

The environment is in cluster and if one indexer is down also , but the delay shouldnt be there .

kindly suggest me what is the root cause so that i check in my environment

Thanks in advance and your answer will be  helpful for me 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

There are several possible causes.

In general you can divide them into three groups:

1) The source reports wrong time (for example - it has a misconfigured clock)

2) The time is not properly parsed from the message (the format is not properly defined, the timezone is off)

3) There is simply a delay between when the event was "produced" by the source and the time it went through all the ingestion process. It can mean that the data is getting stuck somewhere but for some types of sources it can be perfectly normal (like in cases where the batch load of events is synchronized in a cron-scheduled manner from the source or with some cases of WEF subscriptions or some periodicaly working modular inputs). Not every source can deliver events continuously.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

_time is usually derived from the timestamp data found in the event - how have you defined your sourcetype with respect to finding the timestamp?

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...