Why are there two different time formats in the columns?

[tb5821]

Hi - I had splunk import a fairly simple two column file - column 1 was a date/time column2 is some info... the problem seems to be that some of the values in column 1 are in EST and some are in UTC.

I don't think splunk is interpreting these correctly - is there a way I can verify this?

[tb5821]

Data sample

02/07/18 03:55:00 PM EST String=2
02/07/18 03:55:04 PM EST String=3
02/07/18 03:55:08 PM EST String=0
02/09/18 11:10:01 PM UTC String=1
02/09/18 11:10:04 PM UTC String=0
02/09/18 11:10:07 PM UTC String=0

[micahkemp]

So, your sample data already has mixed timezones? Or is that sample data representative of how Splunk parsed it? If the latter, can you include the actual CSV prior to having Splunk handle it?

[tb5821]

Sample data has mixed time zones

[micahkemp]

So when you view the data in Splunk, does the extracted time agree with what your CSV has?

[richgalloway]

If column 1 has time zone information in it ("EST", "UTC", "-0500", "Z", or similar) then your props.conf settings can be tweaked to interpret times correctly. If not, do you have any control over how the file is written?

[tb5821]

Thanks I do have time zone UTC/EST Time zone designators...Guidance on the best way to modify the props file?

[tb5821]

Oh also I thought I read somewhere that Splunk should automatically be able to pick that up from the file? Meaning that the props file does not need to be changed?

[richgalloway]

Automatic doesn't always work like it should, as you've discovered. If you can post some sample events (with private into masked) we can help with the right props settings.

[tb5821]

Thanks - its really a very basic file 🙂 - added sample to the post

[richgalloway]

Try these props.conf settings.

TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%y %H:%M:%S %p %Z
MAX_TIMESTAMP_LOOKAHEAD = 25
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false

[tb5821]

Thanks , do I need to re-index the data too?

[richgalloway]

"reindeer"? Is that auto-correct for re-index? If so, yes, you need to re-index the data for the new props to be applied.

[tb5821]

OK I tried but can't seem to get the formatting to work

_time comes out as
2/6/18 9:27:42.000 AM

where as the time in the file is
2/06/18 04:27:42 PM EST

[richgalloway]

Did you restart Splunk after changing props.conf?