topic Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format. in Getting Data In

Convert DateTime format befor indexing and write in _time. Time of event have hex format.

chernigin_yuri — Mon, 17 Apr 2017 09:58:48 GMT

Hello everybody!
I have trouble with parsing time of event in time indexing.Fields of time in my raw event have hex system - this unix time. How i can to convert from hex to decimal and convert from unix to human readble date time. But i wanna do this before indexing, fore example in props.conf and transforms.conf, i need write down this time in _time

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

richgalloway — Mon, 17 Apr 2017 12:05:01 GMT

Please share some sample data.

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

chernigin_yuri — Mon, 17 Apr 2017 21:34:12 GMT

For example:

<telegram time_formatted="undefined date" time="0x56A7DEC7" type="16" datalen="2001"> // this is the title of each event.

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

Ravan — Tue, 18 Apr 2017 10:09:20 GMT

I tried to convert the time value(0x56A7DEC7) using below link and got the date as "GMT: Tue, 26 Jan 2016 21:01:59".

https://www.epochconverter.com/hex

If the conversion is accurate , you can follow the xml option mentioned in below post.

https://answers.splunk.com/answers/4880/hex-encoded-unix-timestamp.html

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

chernigin_yuri — Tue, 29 Sep 2020 13:43:24 GMT

I tried like this:

https://answers.splunk.com/answers/4880/hex-encoded-unix-timestamp.html

and like this:

https://answers.splunk.com/answers/30852/hex-time-stamp-extraction-issues-with-datetime-config.html

But it did not work out.

Below my configurations:

props.conf

[test_write_hextime_to_timestamp]
DATETIME_CONFIG = /etc/my_hex_epoch_datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 8
TIME_PREFIX = time="
BREAK_ONLY_BEFORE = <telegram
MUST_BREAK_AFTER = </telegram>
REPORT-test-hex-convert = REPORT-test-hex-convert
EVAL-date_time_test = strftime(tonumber(time, 16), "%m:%d:%Y %H:%M:%S")

my_hex_epoch_datetime.xml

<define name="_hexepoch" extract="hexepoch">
    <text><![CDATA[time="0x([\da-fA-F]{8})]]></text>
</define>
<timePatterns>
    <use name="_hexepoch"/>
</timePatterns>
<datePatterns>
</datePatterns>

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

Ravan — Tue, 29 Sep 2020 13:43:30 GMT

I tried with a sample data and it does work. Can double check stanzas and file permissions..?

Here are the configs i have. (Avoid the spaces in the config file line beginnings)

props.conf

[your sourcetype]
TIME_PREFIX = time="
MAX_TIMESTAMP_LOOKAHEAD = 16
BREAK_ONLY_BEFORE = <telegram
MUST_BREAK_AFTER = </telegram>
DATETIME_CONFIG = /etc/my_hex_epoch_datetime.xml

$SPLUNK_HOME/etc/my_hex_epoch_datetime.xml

<datetime>
          <define name="_hexepoch" extract="hexepoch">
            <text><![CDATA[0x([A-Fa-f0-9]{8})]]></text>
          </define>
          <timePatterns>
            <use name="_hexepoch"/>
          </timePatterns>
          <datePatterns>
          </datePatterns>
</datetime>

Re: Convert DateTime format befor indexing and write in _time. Time of event have hex format.

chernigin_yuri — Thu, 20 Apr 2017 13:10:47 GMT

Many thanks!!! Success!
Your configuration is rigth.