we have data written in MST and data is indexed without any issue.
Splunk servers are in CST as well forwarder in CST timezone and we are getting indexed time in CST which is one hour behind of MST so team wants to be indexed in MST rather than in CST. Challenge here is we can't change any TZ for both splunk as well forwarder servers.
Currently no props applied and tried applying US/Mountain but didn't worked. Any suggestions to make indexed time in MST.
[5/5/23 9:33:50:997 MST] 0000000 SystemOut
[5/5/23 9:33:50:994 MST] 0000000 SystemOut O ** ACCESS
@richgalloway , are you talking about forwarder service to be restarted?
Any instance where a .conf file is edited from the CLI needs to be restarted so it will read the changes.
Added TZ = US/Mountain to the exiting props but didn't worked that as well.
Try other time zone designations like America/Denver or MST.
Frankly, some of what you've tried already should have worked. Did you restart Splunk after changing props.conf? Are you looking at newly-onboarded data after each change? Indexed data will not change.
Tried applying props, but didn't worked, same result
TIME_PREFIX = ^\[
TIME_FORMAT = %m/%d/%y %H:%M:%S:%3N %Z
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
Also, your event timestamp seems to be different from the event. Remember the event results display the event from your timezone perpesctive.
So if you're in Rio de Janeiro (UTC-3) and the event contains a UK (UTC-0) timestamp. Your timestamp shows from your timezone perspective (the event timestamp less 3 hours).
Try the TZ = <> attribute in props.conf but also try to change your timezone preference in Splunk to the same TZ that you configured to see if there's any difference.
Now try to apply the TZ = <time_zone> in props.conf
I believe that for your TZ it will end like this:
TZ = America/Boise
See wiki list for all timezones: List of tz database time zones - Wikipedia
Splunk applies time zones in the following order:
1- TZ indicator in the raw event data
2. Value TZ = <value> attribute set in props.conf
3. FW TZ
4. IDX TZ
For the data sample that you provided it seems that your data already contains TZ indicator.
Create a new sourcetype or a props.conf stanza for this host|source for this event containing a timestamp with the TIME_FORMAT = <strptime-style format>
Example:
Data: Valid_Until=Thu Dec 31 17:59:59 GMT-06:00 2020
To extract the timestamp, add this stanza to the props.conf file:
%Z stands for Time zone name
%z stand for UTC offset in the form +HHMM or -HHMM
*In your case, use only %Z
* Use the following table to adapt your data to the strptime format so you can create your own TIME_FORMAT attribute. Python strftime reference cheatsheet
TIME_PREFIX = Valid_Until= TIME_FORMAT = %a %b %d %H:%M:%S %Z%:z %Y
Every sourcetype should have props and lack of props in this case is the reason for the failure. Without props, Splunk has to guess at your data and may guess wrong. Try these props as a starting point
[mysourcetype]
TIME_PREFIX = ^\[
TIME_FORMAT = %m/%d/%y %H:%M:%S:%3N %Z
MAX_TIMESTAMP_LOOKAHEAD = 50
TRUNCATE = 10000
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)