Hello Splunkers!
I am collecting logs from multiple devices, a couple of them have different timezones, so I followed the instructions listed in the following link:
What I did was:
[source::cisco]
TZ = US/Eastern
The timestamp after this change is like this:
Instead of becoming 10:00 AM it did 7:00 AM +3:00
How can this be changed?
Hi @PickleRick
How can I modify the metadata (_time) to match my timezone (the needed timezone)?
For example, my environment is distributed over multiple locations (countries), some assets in USA and some in the middle east, I want to normalize all events to match only 1 timezone.
I hope this makes the situation more clear, please let me know if you have any recommendations.
It depends on your situation. If all your equipment is properly configured and provides a timezone description within the timestamp, you should not need to adjust anything. You should have your TIME_FORMAT set to read the time (including the timezone info) from the event and that's it.
But for it to work the sources must be properly configured and report the time in the proper timezone.
I've seen too many devices which would be misconfigured and simply reported wrong time (for example, were set to UTC and reported UTC as the local time).
Similarily - if your sources were all reporting in UTC, you could just set your TIME_FORMAT without the timezone info and set your TZ for the whole sourcetype to UTC and be done with it - Splunk would parse the time correctly.
But if you have multiple sources which report their time without the timezone info... well, you have to adjust the TZ value for each source separately.
But still - it will not change the contents of the raw event, only what you see as the _time field along with the event.
There is also the issue of what kind of a source you have and how you're ingesting data from it because some processes (like SC4S) might push the timestamp parsed by them and you will not be able to easily adjust it later without doing some heavy mumbo-jumbo like ingest-evals.
You're mistaking two different things - the timestamp contained within the raw event (which is the one you higlighted and which usually should _not_ be tampered with). And the timestamp extracted from said event and stored as a metadata field (shown on the left in your screen).
With an event with a timestamp like this - containing a well-defined timezone information - you should _not_ need to specify a timezone for the source. You should use the timezone provided with the event. It's a matter of a proper TIME_FORMAT setting so that Splunk interprets the timestamp properly. Then Splunk would extract the timestamp from the message and store it along the event (not touching the event contents!) and display it as the contents of the _time field (the one on the left of your screenshot) in the timezone set in your user's preferences.
Anyway, if your device located in your local timezone shows a timestamp offset by the timezone definition, something is wrong. If it's 10AM your local time (GMT+3) and the device sends it as (7AM+3:00), it's wrong. It should send the local timezone (10AM) along with the timezone specification (+3:00).