Splunk Enterprise Security

How to extract time string represented by hex (TA-sav)?

coleman07
Path Finder

The sample data which comes with the TA-sav add-on has its timestamp in a weird hexadecimal format.
It looks like this: 2701140D2636. This number contains 6 hexadecimal octets. The first (27) is hex for the number of
years since 1970, 2nd (01) month, 3rd (day), 4th ( hour), 5th (minute) and 6th (seconds).

Can Splunk, via its transforms and props, convert this date to something which can be indexed or searched? If so, how?

0 Karma

jeffland
SplunkTrust
SplunkTrust

That weird time is epoch time: http://en.wikipedia.org/wiki/Unix_time , measured in milliseconds since 1.1.1970 00:00 UTC.
You could split your hexadecimal value to octets with rex, convert each one to decimal with tonumber(hex) and then multiply them according to how many milliseconds that is (i.e., the first octet is 39 in decimal, multiplied by 31556926 to get 39 years in milliseconds). When you add those together, you get a timestamp in epoch. After you have that, Splunk will display the timestamp in human readable format when you look at it.

0 Karma

coleman07
Path Finder

Looking to see if this can be done at index time.

0 Karma

jeffland
SplunkTrust
SplunkTrust

Hm. I'd like to think it's possible, with a custom datetime extraction: http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition I know you can work with hex data: http://answers.splunk.com/answers/4880/hex-encoded-unix-timestamp.html but I don't know how to go about and actually do it in this case.
What I tried so far is create a sample log file "sample_hex_epoch_log.txt" for testing purposes:

2701140D2636 site:up
270115AF1006 site:up
270115B16EFF site:up

I then set up an xml file "custom_hex_epoch.xml":

<datetime>
    <define name="_hexepoch" extract="hexyear, hexmonth, hexday, hexhour, hexminute, hexsecond">
        <text><![CDATA[([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})([0-9a-fA-F]{2})]]></text>
    </define>
    <timePatterns>
        <use name="_hexepoch"/>
    </timePatterns>
    <datePatterns>
    </datePatterns>
</datetime>

And I added this to props.conf:

[source::sample_hex_epoch_log]
DATETIME_CONFIG = /etc/custom_hex_epoch.xml
MAX_TIMESTAMP_LOOKAHEAD = 12

But when I try to import a sample log, it says "failed to parse timestamp". I'm unsure about the regex and about the rest of the xml though, maybe someone else can help shed some light here?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.


Introducing Unified TDIR with the New Enterprise Security 8.2

Read the blog

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...