Unable to change timezone of the logs

[ppanchal]

We have a host sending logs in UTC timezone and we want to display it in US/Central timezone.
I have added the below configuration in the props.conf file on our indexer, but this does not help.

[host::(name of the host)]
TZ = US/Central

Where do I need to edit the props.conf file? Search head? Indexer? Deployment server?

Can somebody please assist?

[nickhills]

Can you post an example log message including the timestamp from the original event?
What timezone is the originating server in?

[ppanchal]

URL: /restconnect/connect/users/2770........

Timestamp: 2017-12-20T15:28:55.449Z

_time: 2017-12-20 09:28:55.449

[nickhills]

I am confused, If I understand your response, the raw log says 15:28:55Z (ie UTC)
but _time (by which i assume you mean the timestamp Splunk is reporting) says 09:28:55 is correctly adjusted -6 hours?

[ppanchal]

You are correct.

[nickhills]

Cool, Glad its sorted.
Please accept one of the answers/upvote if I helped you - It helps future visitors know that we got to the bottom of it 🙂

[ppanchal]

No its not sorted, we want to make the _time as 15:28:55Z instead of 09:28:55,
The timestamp and _time should be same and in Central timezone.please help.

[nickhills]

Ok, then I am still confused 🙂

Your original question said:
"We have a host sending logs in UTC timezone and we want to display it in US/Central timezone."

Thats exactly what you have right now.

Splunk won't (can't) update the _raw log data, which seems to be what you are asking.

The only way i can reason this out in my mind, is that you are saying the time in the original log data is wrong.

[nickhills]

What if you set your user preferences to UTC - this would display both values as 15:28? (but would probably screw up any other events)

[ppanchal]

I would like to re frame here,

Timestamp: 2017-12-20T15:28:55.449Z (This is already displayed as CST)

_time: 2017-12-20 09:28:55.449 (This is UTC)

I want to convert _time to CST.

I hope it helps now.

[nickhills]

I'm sorry dude, but your wrong on both fronts.

The Z in the timestamp specifically means the time recorded is in ZULU time, or UTC. Not CST
https://stackoverflow.com/questions/9706688/what-does-the-z-mean-in-unix-timestamp-120314170138z

Even if that was not the case.. UTC is not behind CST. The uk is 6 hours ahead of central US.

This means that the event was recorded at 3:28 in the afternoon UTC - regardless of where you happen to be - Since you are (i assume) in Central USA, 3:28 PM in the uk, is 09:28AM where you are.

I don't think you have a config issue - we can even prove it if you like.
Do a realtime search for these events - My 50cent bet says you will see events popping into the right side of your timeline, meaning they are arriving "now" - the raw log message will say 19:35(ish if your online when i send this) but your _time will be 13:35 which i think is the time where you are right now.

[ppanchal]

This is so confusing, not sure what the issue is.

My raw log says timestamp: 2017-12-21T14:06:08.893Z

My _time says 21/12/2017 08:06:08.893

My machine is set to CST.

User preferences is also set to CST.

[nickhills]

🙂
We will get to the bottom of this!

Run this over the last 15 minutes and paste the first few rows of the table.

<your search> |eval indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S")|table indextime _time _raw |sort -indextime

[ppanchal]

Ok so if I search for last 15 mins, I do not see any logs.

But when I search for today, this is what I see, image uploaded

https://ibb.co/d5469m

[nickhills]

Do you still have this in props?

[host::(name of the host)]
TZ = US/Centra

Also - Where is a.) your splunk server b.) your server producing the logs - Are they both systems you manage, or are they remotely hosted?

[ppanchal]

[host::(name of the host)]
TZ = US/Centra
I removed this from my logs today. Do you want me to add them? If yes, then where search head, indexer or deployment?

a) By splunk server if you mean search head, indexer or deployment server then yes I manage them.

b) the server producing logs is remotely hosted

[nickhills]

it will only make a difference on the indexer or heavy forwarders.
Lets make sure its removed - Can you confirm that you have removed
TZ = US/Central from any indexers and hfs and restarted them?

The remote server:
- is its managed by a third party
- List item
- can you manually get logs from it
- how do you get logs - UF collecting files, or UF running a script - something else?

[ppanchal]

Yes its removed and servers are restarted, confirmed.

[nickhills]

Its generally a good idea to index the events in the correct timezone, or else you are starting down a painful road.

If you want to see these in your local timezone - update your user preferences to specify which TZ you are in, and splunk will adjust how it renders them for you.

[nickhills]

From the Splunk UI, click your username in the top right bar.
Select account Settings.
Set your timezone.

[ppanchal]

It is already set to Central still we see the logs in UTC.