topic DB Connect - Timezone Issue: Best Practices to avoid/identify? in Getting Data In

DB Connect - Timezone Issue: Best Practices to avoid/identify?

chawagon03 — Thu, 09 Jul 2015 02:00:33 GMT

I'm going to explain this the best way possible. Our server running the indexer is located in EST. We are querying a database located in a different timezone and timestamping against epoch time, but the indexer seems to get the timestamp wrong and add 4 hours to it. Is there something we can do to make it use GMT time? What do you guys do to prevent timezone issues? Looking for best practices.

Re: DB Connect - Timezone Issue: Best Practices to avoid/identify?

woodcock — Thu, 09 Jul 2015 04:10:54 GMT

The best best-practice is this:

1: Always run host OS in GMT.
2: Always timestamp source data in GMT.
3: Always use NTP.

In a perfect world, everybody would do that and things would be fine. Short of that:

1: Always explicitly timestamp (do not use DATETIME_CONFIG=CURRENT nor DATETIME_CONFIG=NONE).
2: Always timestamp source data with per-event TZ.
3: When that is not possible, always set a TZ= value in props.conf for everything (this can be verified by checking date_zone; it should NEVER be local but it is OK if it is null for some TA inputs).
4: Always keep an eye on your calculated lag ( _indextime - _time) so you see when things go sideways.
5: Always trackdown the reason any events have timestamp=none.

It sounds like you are having a type-3 problem.

Re: DB Connect - Timezone Issue: Best Practices to avoid/identify?

chawagon03 — Thu, 09 Jul 2015 17:47:55 GMT

Thank you! This was great and will hopefully solve future issues!