How to assign custom JSON field with epoch time as the timestamp for events?

[akhanVG]

We are inputting JSON fields to splunk. One of the fields eventTime should be the event time for the index.

{
    browserType:  Mozilla/5.0 (iPhone; CPU iPhone OS 8_1_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B466 Safari/600.1.4 
    campaignLocation:  null 
    campaignName:  null 
    currentPage:  /test
    eventBy:  application 
    eventName:  pageLoad 
    eventSource:  frontend 
    eventTime:  1424822395 
    ipAddress:  127.1.1.1:45770 
    isMobile:  true 
    referrer:  http://tfdf.dfdf.com
    sessionId:  null 
    userId:  null 
}

The eventTime is in millisecond format (UTC)

This is what the JSON value looks like raw in the log file

{"browserType":"Mozilla/5.0 (Linux; Android 5.0; SM-G900V Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.109 Mobile Safari/537.36","campaignLocation":null,"campaignName":null,"currentPage":"/dfd6","eventBy":"application","eventName":"pageLoad","eventSource":"frontend","eventTime":"1424822393","ipAddress":"192.168.1.1:58674","isMobile":true,"referrer":"http:dfsdf,"sessionId":null,"userId":null}

[s2_splunk · Accepted solution]

OK, your complete props.conf for this sourcetype as it exists on the indexer needs to look like this (assuming every event is on its own line):

[hermes]
TIME_PREFIX=eventTime:\s+
TIME_FORMAT=%s
KV_MODE=json

If your timestamp is not in fact an epoch time, but milliseconds, try
TIME_FORMAT=%s%3N

If you can't get it to work, I suggest you use the DataPreview tool in the Splunk UI, but this is very straightforward.

[s2_splunk]

Well, I just noticed your raw json up there, so the prefix (of course) needs to be:

    TIME_PREFIX=\"eventTime\":\"

[akhanVG]

Awesome. Should have added the raw JSON.

This worked. Also I did not have it in EPOCH derp but millisecond so %s%3N worked beautiful.

[s2_splunk]

You had the raw message up there, I just didn't see it. Please accept the answer, if all works now.

[s2_splunk]

TIME_PREFIX = eventTime:\s+

Docs on Timestamp Extraction

[akhanVG]

No dice. Is this supposed to go into the inputs.conf file on the forwarder or some place else?

[s2_splunk]

Line breaking and timestamp extraction happen on the indexer, so this needs to go in props.conf for your sourcetype and be available on the indexer.

Where do I configure my Splunk settings?

[akhanVG]

Yep - did but no dice

[rule::timestamp_hermes]
sourcetype = hermes
TIME_PREFIX = eventTime:\s+

is what I appended to the end of props.conf in /opt/splunk/etc/system/local/

Still doesn't appear to pick up the time.

[s2_splunk]

That's not what that's supposed to look like. Try this (the backslash in the RegEx is important, you had removed it):

[hermes]
TIME_PREFIX= eventTime:\s+

[akhanVG]

Yes that is what I have - still no good. I evne tried replacing hermes (the sourcetype) with ecom (the index name) and still no luck.