Getting Data In

Why is some data source indexed one hour in the future?

emallinger
Communicator

Hi,

Some data source is indexed one hour in the future (probably since TZ shift => twice a year hour change in France !! this time +0100hour).

We were on gmt+1, now we're on gmt+2.


I don't know where the problem is.
- checked the server ntp => ok, gmt+2 updated
- checked the data source file => ok
- tried to reproduced in dev env on a mono-instance : issue not reproducted !

- this is the only data source with the issue

My prod env is distributed (SHC, Indexer Cluster and multiple forwarders)

- data is a jsonl file.

I'm soo lost !!

Thank you for your help !!

Ema

on the indexer cluster :

[mysourcetype]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_PREFIX= "dte":"
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TRUNCATE = 0
MAX_DAYS_AGO = 4000
category = Structured
disabled = false
pulldown_type = true

 

data sample :

{"idj":"3108824152","dce":"IDN","fce":"IDN2","ace":"176","dte":"08/04/2022 14:44:31","org":"GN","dmc":"2","idu":"211151","csu":"00082827","lsu":"CROSS BDOHRIJ GHBGD14 ","ctx":"Identifiant:PN-003042021007790-ARD-PPM-70732201#Procédure de référence:CIAHTDT CENTRAL DE CNJAEN-2021-007790#Type personne:Physique#Qualité personne:Mise en cause#Nom:XXX#Prénom:yyy#Lieu de naissance:CAEN#Date de naissance:05/01/1991#","idd":"PN-0030428541021007790-ARD-PPM-7074532201","ise":"N","cts":[{"idj":"3108824152","nom":"XXX","pre":"yyy","jne":"5","mne":"1","ane":"1981","lne":"CAEN","cot":"","not":"","qot":"","nuo":"","ctt":"","gtt":"","qtt":"","ntt":""}]}

 

This data is indexed at  08/04/2022 15:44:31 for 08/04/2022 14:44:31 !

 

 

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Your source doesn't report the timezone so the indexer/hf, since no tz is defined for the sourcetype, interprets with its local timezone.

0 Karma

emallinger
Communicator

Hello,

 

Yes, but source and indexer are in the same time zone, at the same hour.

So why the difference ?

Thanks for your suggestions,

Ema

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...