Getting Data In

Why is some data source indexed one hour in the future?

emallinger
Communicator

Hi,

Some data source is indexed one hour in the future (probably since TZ shift => twice a year hour change in France !! this time +0100hour).

We were on gmt+1, now we're on gmt+2.


I don't know where the problem is.
- checked the server ntp => ok, gmt+2 updated
- checked the data source file => ok
- tried to reproduced in dev env on a mono-instance : issue not reproducted !

- this is the only data source with the issue

My prod env is distributed (SHC, Indexer Cluster and multiple forwarders)

- data is a jsonl file.

I'm soo lost !!

Thank you for your help !!

Ema

on the indexer cluster :

[mysourcetype]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_PREFIX= "dte":"
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TRUNCATE = 0
MAX_DAYS_AGO = 4000
category = Structured
disabled = false
pulldown_type = true

 

data sample :

{"idj":"3108824152","dce":"IDN","fce":"IDN2","ace":"176","dte":"08/04/2022 14:44:31","org":"GN","dmc":"2","idu":"211151","csu":"00082827","lsu":"CROSS BDOHRIJ GHBGD14 ","ctx":"Identifiant:PN-003042021007790-ARD-PPM-70732201#Procédure de référence:CIAHTDT CENTRAL DE CNJAEN-2021-007790#Type personne:Physique#Qualité personne:Mise en cause#Nom:XXX#Prénom:yyy#Lieu de naissance:CAEN#Date de naissance:05/01/1991#","idd":"PN-0030428541021007790-ARD-PPM-7074532201","ise":"N","cts":[{"idj":"3108824152","nom":"XXX","pre":"yyy","jne":"5","mne":"1","ane":"1981","lne":"CAEN","cot":"","not":"","qot":"","nuo":"","ctt":"","gtt":"","qtt":"","ntt":""}]}

 

This data is indexed at  08/04/2022 15:44:31 for 08/04/2022 14:44:31 !

 

 

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Your source doesn't report the timezone so the indexer/hf, since no tz is defined for the sourcetype, interprets with its local timezone.

0 Karma

emallinger
Communicator

Hello,

 

Yes, but source and indexer are in the same time zone, at the same hour.

So why the difference ?

Thanks for your suggestions,

Ema

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...