How to convert epoch time with milliseconds into splunk at indexing time

[vrmandadi]

I have a file that I am monitoring has time in epoch format milliseconds .What setting should be placed in the props.conf to convert it to human readable

[manjunathmeti]

Use INGEST_EVAL in transforms.conf on indexers:

props.conf

[mysourcetype]
TRANSFORMS = myeval

transforms.conf

[myeval]
INGEST_EVAL = human_readable_field = strftime(epoch_field_from_data, "%m-%d-%Y %H:%M:%S.%3N")

And on search heads add this field in fields.conf so that users can search this field.
fields.conf

[human_readable_field]
INDEXED = True

[richgalloway · Accepted solution]

You don't want to convert timestamps to human-readable format at index time because a human is not reading the timestamp at index time. Use TIME_FORMAT = %s%3N to tell Splunk the timestamp is in epoch form with milliseconds. Remember to set TIME_PREFIX properly.

Do the conversion to human-readable format at search time. Do so using fieldformat as late as possible in the query.

[HattrickNZ]

%s is for 10 digits epoch? I have a 9 digit epoch and it seems the %s is not bsorking forb me?

[inventsekar]

@HattrickNZ if you are having a 9 digit epoch means, thats timestamp value is a very old timestamp(before Sun Sep 9 01:46:39 2001 UTC)(but ideally it should work fine, something wrong.. pls provide us your search query)

Question

Will UNIX's epoch time change from 9 to 10 digits affect Gentran? (SCI8237)

Answer

No. epoch time is how time is kept track of internally in UNIX. It's seconds, counting upward from January 1st, 1970. This number hit 1 million (1,000,000) in March of 1973, and will hit one billion (1,000,000,000) on Sun Sep 9 01:46:39 2001 UTC. This change, from a number which can be represented in 9 decimal digits to a 10-digit number, is not expected to cause any problems for UNIX systems.

The reason is that this value is not stored as decimal digits. Instead, it is stored as an integer value (a 32-bit binary variable) which can be used safely until the year 2038, when the epoch date goes back to 0. The uses in UNIX of a decimal format for the "seconds time" value are primarily in portable file formats, such as tar, cpio, and ar. These formats have always supported at least eleven decimal (or octal in some cases) digits, easily handling UNIX's one-billionth "birthday".

https://www.ibm.com/support/pages/will-unixs-epoch-time-change-9-10-digits-affect-gentran-sci8237-st...

[vrmandadi]

Thank you @richgalloway .What time format do I need to set for events which have Mar 25, 21:43 UTC as timestamp

[richgalloway]

%b %d, %H:%M:%S %Z. See the "Date and time format variables" section of the Search Reference manual.

[vrmandadi]

Thank You.