Splunk Enterprise

Why is there a delay in index time and search time data?

dntest
New Member

Delay in index time and search time data..
There is a delay of 10 hours 

index=test_shift  "*10987867*" | eval indextime=strftime(_indextime,"%d/%m/%Y %H:%M:%S")
| table _raw _time indextime



_time is 2022-11-15 13:42:31
indextime is 2022-11-15 23:27:33

The environment is in cluster and if one indexer is down also , but the delay shouldnt be there .

kindly suggest me what is the root cause so that i check in my environment

Thanks in advance and your answer will be  helpful for me 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

There are several possible causes.

In general you can divide them into three groups:

1) The source reports wrong time (for example - it has a misconfigured clock)

2) The time is not properly parsed from the message (the format is not properly defined, the timezone is off)

3) There is simply a delay between when the event was "produced" by the source and the time it went through all the ingestion process. It can mean that the data is getting stuck somewhere but for some types of sources it can be perfectly normal (like in cases where the batch load of events is synchronized in a cron-scheduled manner from the source or with some cases of WEF subscriptions or some periodicaly working modular inputs). Not every source can deliver events continuously.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

_time is usually derived from the timestamp data found in the event - how have you defined your sourcetype with respect to finding the timestamp?

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

 Prepare to elevate your security operations with the powerful upgrade to Splunk Enterprise Security 8.x! This ...

Get Early Access to AI Playbook Authoring: Apply for the Alpha Private Preview ...

Passionate about security automation? Apply now to our AI Playbook Authoring Alpha private preview ...

Reduce and Transform Your Firewall Data with Splunk Data Management

Managing high-volume firewall data has always been a challenge. Noisy events and verbose traffic logs often ...