I have a log that is putting the timestamp in UTC without any timezone, which appears Splunk thinks this is my time and I cannot get it to understand that the timezone is UTC
Example log:
Feb 21, 2021 00:03:05.973 [0x7f6a027c5700] DEBUG - Completed: [127.0.0.1:42302] 206 PUT ...
In my `$SPLUNK_HOME/etc/apps/search/default/props.conf` I have added:
[default]
TZ=GMT
My timezone is set in preferences to -0800 (Pacific) but it still shows these as the same time.
I agree with @richgalloway that you should not assign time in default.
Please review the precedence of configuration files: https://medium.com/splunkuserdeveloperadministrator/splunk-configuration-files-precedence-explained-...
You likely have something overriding the timezone in your app local or system local directory. I would check the props where the sourcetype is defined.
Additionally, if this data is forwarded, the indexer may not parse and change the time unless you specifically tell it to go to the parsing queue. Forwarded data will not always go through each queue that data would go through in a single-instance deployment.
https://wiki.splunk.com/Community:HowIndexingWorks
Often times when I have a item not being applied from props, I find it is because I have the props stanza "applied" after the data has gone through the parsing queue, so it is not really applied.
I nuked my /etc files which caused Splunk to recreate. Didn't fix anything, so that wasn't the issue. Its not the indexed data issue the Splunk search is always in Pacific, no matter what preference I have set up
For example, it is 12am in CA and Splunk search thinks my user is in UTC when its set to Pacific.
First, NEVER edit a default config file. All changes to config files must be made in a 'local' directory. Create the file, if necessary.
Second, don't worry about this until you fix your system timestamp problem.