Getting Data In

How to configure Splunk to set _time to a specific field?

Ultra Champion

We ended up using the following -

base search 
| eval _time=strptime(eventStartTime,"%Y-%m-%d %H:%M:%S.%N")

Which works perfectly.

Is there a way to set it up in the configuration, so eventStartTime is assigned to _time?

0 Karma
1 Solution

Builder

If all events from this source have eventStartTime you can setup a props.conf setting for that source/sourcetype that tells splunk what timestamp to use when assigning the _time value.

Based on the event you provided , and assuming that your events are not multi-lined, you could add this to your indexers props.conf.

[source::/opt/log/yourLog]
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX=eventStartTime=
SHOULD_LINEMERGE = false

View solution in original post

Explorer

Hello,

I am trying to use a timestamp that is NOT _time. My time stamp is Transaction_Date. When I click the small magnifying glass near the timepicker in the search command, I am getting all results , which means it picking _time and Not Transaction date. I tried all the below commands and still not working. can you please help.

index=base search... | eval  _time=strptime(Transaction_Date,"%Y-%m-%d %H:%M:%S.%N")

I just want the magnifying glass to pick Transaction_Date timestamp and Not _time originally indexed.

0 Karma

Ultra Champion

An event looks like -

level=WARN [http-/xxx.xx.xxx.xxx:8080-1]|logTime=2016-04-28 12:51:57,250 UTC |e2eElapsedTime=3|protocol=http|ext.request.header.x-forwarded-server=xxxxx.xxxx.xxxx.com|host=xxxxxx.xxx.com/10.106.204.36|ext.request.header.content-length=null|operationStatus=SUCCESS|ext.request.header.postback-url=null|eventStartTime=2016-04-28 07:51:57.246|ext.request.header.x-forwarded-host=xxx.xx.xxx.xxx/xxx.xx.xxx.xxx|ext.request.header.user-agent=Apache-HttpClient/4.5 (Java/1.7.0_80)|destination=http://xxxx.xxx.xxx.com/xxxx/v1.0/responsestatus/xxxxxx/ef032e5b-39d0-49d8-b31a-0407f5e67926|eventElapsedTime=3|operationDetail=SUCCESS-200-SUCCESSFUL-OK|node=56a126c8f3d723a3c6004070|environment=xxxxx|ext.request.header.content-type=application/json|transID=|component=cisl|external_correlation_id=null|ext.request.header.x-client-ip=10.237.76.72/10.237.76.72|ext.request.header.clientid=null|src=10.237.76.72|activity=GET->/xxxx/v1.0/responsestatus/clinicalauthorizations/ef032e5b-39d0-49d8-b31a-0407f5e67926

So, logTime appears before eventStartTime.

0 Karma

SplunkTrust
SplunkTrust

It would help to see some sample events. In general, however, you probably need to adjust the settings for that sourcetype in your props.conf file. There may be another timestamp prior to eventStartTime that Splunk is finding and using for _time. In particular, check out the TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD, TIME_FORMAT attributes.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Builder

If all events from this source have eventStartTime you can setup a props.conf setting for that source/sourcetype that tells splunk what timestamp to use when assigning the _time value.

Based on the event you provided , and assuming that your events are not multi-lined, you could add this to your indexers props.conf.

[source::/opt/log/yourLog]
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX=eventStartTime=
SHOULD_LINEMERGE = false

View solution in original post

Ultra Champion

Really great. I just made the change and restarted Splunk. Is there a "confirmation", in a sense, in a log file, for example, which acknowledges that the config change takes place?

0 Karma

Ultra Champion

qq, if the eventStartTime field doesn't exist in the event, what would be the behavior?

0 Karma

Builder

Taking a guess here but I believe it would use the time of the previous event. Think you would see a warn message like this

05-09-2016 18:57:01.213 +0000 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Mon May 9 18:57:00 2016)

0 Karma

Revered Legend

This should be configured during the event ingestion. COntact your Splunk Admin to set the timestamp parsing for the sourcetype that you're using, to permanently assign eventStartTime to _time.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!