Getting Data In

Why is some data source indexed one hour in the future?

emallinger
Communicator

Hi,

Some data source is indexed one hour in the future (probably since TZ shift => twice a year hour change in France !! this time +0100hour).

We were on gmt+1, now we're on gmt+2.


I don't know where the problem is.
- checked the server ntp => ok, gmt+2 updated
- checked the data source file => ok
- tried to reproduced in dev env on a mono-instance : issue not reproducted !

- this is the only data source with the issue

My prod env is distributed (SHC, Indexer Cluster and multiple forwarders)

- data is a jsonl file.

I'm soo lost !!

Thank you for your help !!

Ema

on the indexer cluster :

[mysourcetype]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_PREFIX= "dte":"
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TRUNCATE = 0
MAX_DAYS_AGO = 4000
category = Structured
disabled = false
pulldown_type = true

 

data sample :

{"idj":"3108824152","dce":"IDN","fce":"IDN2","ace":"176","dte":"08/04/2022 14:44:31","org":"GN","dmc":"2","idu":"211151","csu":"00082827","lsu":"CROSS BDOHRIJ GHBGD14 ","ctx":"Identifiant:PN-003042021007790-ARD-PPM-70732201#Procédure de référence:CIAHTDT CENTRAL DE CNJAEN-2021-007790#Type personne:Physique#Qualité personne:Mise en cause#Nom:XXX#Prénom:yyy#Lieu de naissance:CAEN#Date de naissance:05/01/1991#","idd":"PN-0030428541021007790-ARD-PPM-7074532201","ise":"N","cts":[{"idj":"3108824152","nom":"XXX","pre":"yyy","jne":"5","mne":"1","ane":"1981","lne":"CAEN","cot":"","not":"","qot":"","nuo":"","ctt":"","gtt":"","qtt":"","ntt":""}]}

 

This data is indexed at  08/04/2022 15:44:31 for 08/04/2022 14:44:31 !

 

 

Labels (1)
0 Karma

PickleRick
Ultra Champion

Your source doesn't report the timezone so the indexer/hf, since no tz is defined for the sourcetype, interprets with its local timezone.

0 Karma

emallinger
Communicator

Hello,

 

Yes, but source and indexer are in the same time zone, at the same hour.

So why the difference ?

Thanks for your suggestions,

Ema

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...