topic Re: Using props.conf to change timestamp in Getting Data In

Using props.conf to change timestamp

jedatt01 — Mon, 28 Sep 2020 16:17:21 GMT

I have a modular input that collects data from a webservice. The events are not collected in realtime so to get the true timestamp I have to extract that from the time field on each event instead of using when splunk consumes it as the time stamp.

The problem with this is the event time is in UTC and my server is in US/Eastern time (UTC-5). When I search for the events they show 5 hours ahead. This causes problems when using relative search times because no data shows up.

How can I use props.conf or other method to make the events show up in Splunk as US/Eastern time so my searches work correctly? My current props.conf is below. I've tried to change the TZ= setting but it makes no difference. Please help!

[test]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^{
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q
TZ=UTC
KV_MODE=json
TRUNCATE=15000

Here's what the raw event logs like from splunkd.log
{
'tot': 86,
'epoch': 1396352800,
'tos': 85,
'sid': 318,
'browsertype': IE7,
'type': 'txtest',
'sname': New York, NY - Verizon,
'ttime': 2014-04-01 07:46:40.433,
'tpf': 0,
'rtime': 5954,
'nbyte': 729580,
'tof': 0,
'mid': 14247945,
'tps': 3, 'tpt': 3
}

Re: Using props.conf to change timestamp

aelliott — Tue, 01 Apr 2014 14:36:00 GMT

The timezone will be applied on index time. Therefore you cannot modify existing data to show correctly. You may want to export the data and re-import it.

Re: Using props.conf to change timestamp

jedatt01 — Tue, 01 Apr 2014 15:07:23 GMT

What about data that has not already been indexed? I'm not concerned about the data that is already there.

Re: Using props.conf to change timestamp

Ayn — Tue, 01 Apr 2014 15:09:55 GMT

Where did you configure this, on the indexer?

Re: Using props.conf to change timestamp

jedatt01 — Tue, 01 Apr 2014 15:19:17 GMT

This is a single server splunk deployment. The events come directly into the splunk server via webservice.

Re: Using props.conf to change timestamp

linu1988 — Tue, 01 Apr 2014 18:49:29 GMT

Hello,
I checked the data. Your intended timestamp was not recognized. Try the below configuration.

[test] 
NO_BINARY_CHECK=1 
SHOULD_LINEMERGE=true 
BREAK_ONLY_BEFORE=^{ 
TIME_PREFIX='ttime':
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q 
TZ=UTC 
KV_MODE=json 
TRUNCATE=15000

Thanks

Re: Using props.conf to change timestamp

jedatt01 — Tue, 01 Apr 2014 19:10:22 GMT

That did the trick! thanks