Getting Data In

Coverting raw IIS logs timestamp from GMT to EST

Jarohnimo
Builder

Hello All,

I'm working in a windows environment ingesting IIS logs from windows servers. The logs are written in GMT and i'd like to convert the raw GMT Time stamp to Easter Standard time. Perhaps it can happen at Index time or if there's a way to do this through transforms.conf or props please let me know (and how).

Currently Splunk's timestamp of the log is in Eastern Standard time zone however within the raw logs when searching sourcetype=iis the logs time stamp shows as GMT. I'd prefer to have that raw log match Splunk's time stamp and would like to know what's the best way to go about converting it.

I don't want to convert it at search time as i want the change to be for anyone who uses splunk (why i'd like a configuration file solution).

thanks for your support.

0 Karma
1 Solution

Jarohnimo
Builder

It ended up being an issue with the file permissions on my props.conf on my indxer. I corrected it and problem was solved

View solution in original post

0 Karma

Jarohnimo
Builder

It ended up being an issue with the file permissions on my props.conf on my indxer. I corrected it and problem was solved

0 Karma

koshyk
Super Champion

There are two aspects to your query
1. How much integrity of your data is important? In our case, we need the pure/raw data as it is and shouldn't manipulate data. That means the data as it is will be in GMT, but Splunk display time of the event will be in your local EST. Best practice in many organisations especially global orgs
2. You can manipulate the raw data at indextime. This may be complex and need reconfig on how it is displayed.

Another good reference link is : https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Applytimezoneoffsetstotimestamps
For Option1, above you just need to specify the UserProfile for timezone information in your settings

If you want to try out indextime settings , Have a try using

[yoursourcetype]
TZ = US/Eastern

Advanced options are

[<spec>]
DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>
TIME_PREFIX = <regular expression>
MAX_TIMESTAMP_LOOKAHEAD = <integer>
TIME_FORMAT = <strptime-style format>
TZ = <POSIX time zone string>
MAX_DAYS_AGO = <integer>
MAX_DAYS_HENCE = <integer>
MAX_DIFF_SECS_AGO = <integer>
MAX_DIFF_SECS_HENCE = <integer>
0 Karma

Jarohnimo
Builder

So it's sounding like I want what you've done at your organization. The main thing is that all time fields match and show as Eastern standard Time zone. We only care about how it's displayed in splunk, not necessarily how it's writes the raw data to the buckets at index time.

Perhaps because I have set my time zone up under settings it's showing correctly for _time however I'd like the settings for both _time + time field in IIS to both show as Eastern standard time zone (or their local time as you've specified) for everyone so no one has to adjust their settings individually

What would I need to adjust to accomplish that? Props? Transforms?

I noticed when I do a search for iis logs between 24 hours in the future and 3hours it returns results as splunk is a bit confused with GMT and me adjusting my personal settings to est

In a separate environment I haven't adjust my personal settings for tz and it's results are no logs are from the future... The fix I believe is what you've applied in your environment
Thanks for your help

0 Karma

koshyk
Super Champion

I'm not entirely sure about the requirements, but have a try in your development/test environment to set user profile and local settings

https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Userlanguageandlocale

Have a try and see if this is what you expected, by searching the logs/events. The time should be displayed nearby your event. If this solves, it may be easier than trying in props/transforms

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...