topic Re: How to prevent Splunk from converting time to date in Getting Data In

How to prevent Splunk from converting time to date

indeed_2000 — Mon, 09 Dec 2019 17:37:35 GMT

Hi,
I have a log file like this:

08:00:00.032 user     parameter: A[0]B[0]C: Action successful.

This is just hour:minutes:seconds:mile seconds
As you see the only time of that event exists in log, and Splunk _time automatically converts this 08:00:00.032 to timestamp!
This is the cause of the wrong date for events.

For example, If I added today's log file to Splunk it will show events belong to 2018 ,2017, 2016 ...

Any recommendation?
Thanks.

Re: How to prevent Splunk from converting time to date

richgalloway — Mon, 09 Dec 2019 18:31:08 GMT

Do you have TIME_FORMAT = %H:%M:%S.%3N in your props.conf?

Re: How to prevent Splunk from converting time to date

indeed_2000 — Mon, 09 Dec 2019 19:33:11 GMT

1-Add this props.conf
2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

Re: How to prevent Splunk from converting time to date

richgalloway — Mon, 09 Dec 2019 22:06:51 GMT

What should the correct timestamp be?

Re: How to prevent Splunk from converting time to date

adonio — Tue, 10 Dec 2019 03:26:53 GMT

you can use the _time to be the index time in props.conf

 [mysourcetype]
 DATETIME_CONFIG = CURRENT

now you have 2 time fields, _time and your field with HMS.MS

hope it helps

Re: How to prevent Splunk from converting time to date

indeed_2000 — Tue, 10 Dec 2019 07:17:58 GMT

should I do something else after this change? e.g. restart splunk service?

Re: How to prevent Splunk from converting time to date

indeed_2000 — Tue, 10 Dec 2019 07:20:35 GMT

Hour:Minutes:Seconds:Millisecond
08:00:00.122 = 08 AM

Re: How to prevent Splunk from converting time to date

FrankVl — Tue, 10 Dec 2019 07:42:50 GMT

It is not very clear what your issue is here. Since the timestamp does not include a date, I believe Splunk will assume the date is today, and index all events as such.

Do you have logs from multiple dates in your file, but without a date in the events? Or what exactly is your issue? Maybe show what the _time field looks like in Splunk and explain what is wrong with it.

Re: How to prevent Splunk from converting time to date

indeed_2000 — Tue, 10 Dec 2019 12:34:16 GMT

1-file contain only log of today.
2-there is no date in log it just time 08:00:00.122
3-"_time" also show logs that belong this dates 2018 ,2017, 2016 ...

exact problem is splunk convert time to date e.g.

source="/opt/logs-20191210.log" | table _time

1   2018-02-02 09:04:04.042
2   2018-02-02 09:04:04.041
3   2018-02-02 09:04:04.041
4   2018-02-02 09:04:04.039
5   2018-02-02 09:04:04.039

....

1   2017-07-13 08:43:56.928
2   2017-07-13 08:43:56.927
3   2017-07-13 08:43:56.925
4   2017-07-13 08:43:56.925
5   2017-07-13 08:43:56.920

....

1   2016-12-26 08:48:35.986
2   2016-12-26 08:48:35.986
3   2016-12-26 08:48:35.984
4   2016-12-26 08:48:35.979
5   2016-12-26 08:48:35.979

...

1-here is the props.conf

[logs-20191210-too_small]
TIME_FORMAT = %H:%M:%S.%3N

also try this one

DATETIME_CONFIG = CURRENT

2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

Re: How to prevent Splunk from converting time to date

FrankVl — Tue, 10 Dec 2019 14:49:40 GMT

Where did you deploy that props.conf? It should be on the first full splunk enterprise instance that processes the data (so on a HF or Indexer), not on a universal forwarder. Because if even DATETIME_CONFIG = CURRENT isn't working, then it sounds like that props.conf is not taking effect.

Re: How to prevent Splunk from converting time to date

indeed_2000 — Tue, 10 Dec 2019 15:19:03 GMT

Logs and enterprise splunk locate on the single server and I just want to do this config affect to specific log. Not whole logs.

Re: How to prevent Splunk from converting time to date

FrankVl — Tue, 10 Dec 2019 16:00:07 GMT

What do you mean by that? Are you using some transforms to assign this logs-20191210-too_small sourcetype to these events?

If so: that will not work like this. Only indextime config for the original sourcetype will be applied. You can't override the sourcetype and then apply different indextime config based on that new sourcetype value.

Re: How to prevent Splunk from converting time to date

indeed_2000 — Fri, 13 Dec 2019 16:34:14 GMT

There is no transforming here.