Getting Data In

How to prevent Splunk from converting time to date

Explorer

Hi,
I have a log file like this:

08:00:00.032 user     parameter: A[0]B[0]C: Action successful.

This is just hour:minutes:seconds:mile seconds
As you see the only time of that event exists in log, and Splunk _time automatically converts this 08:00:00.032 to timestamp!
This is the cause of the wrong date for events.

For example, If I added today's log file to Splunk it will show events belong to 2018 ,2017, 2016 ...

Any recommendation?
Thanks.

0 Karma

Ultra Champion

It is not very clear what your issue is here. Since the timestamp does not include a date, I believe Splunk will assume the date is today, and index all events as such.

Do you have logs from multiple dates in your file, but without a date in the events? Or what exactly is your issue? Maybe show what the _time field looks like in Splunk and explain what is wrong with it.

0 Karma

Explorer

1-file contain only log of today.
2-there is no date in log it just time 08:00:00.122
3-"_time" also show logs that belong this dates 2018 ,2017, 2016 ...

exact problem is splunk convert time to date e.g.

source="/opt/logs-20191210.log" | table _time

1   2018-02-02 09:04:04.042
2   2018-02-02 09:04:04.041
3   2018-02-02 09:04:04.041
4   2018-02-02 09:04:04.039
5   2018-02-02 09:04:04.039

....

1   2017-07-13 08:43:56.928
2   2017-07-13 08:43:56.927
3   2017-07-13 08:43:56.925
4   2017-07-13 08:43:56.925
5   2017-07-13 08:43:56.920

....

1   2016-12-26 08:48:35.986
2   2016-12-26 08:48:35.986
3   2016-12-26 08:48:35.984
4   2016-12-26 08:48:35.979
5   2016-12-26 08:48:35.979

...

1-here is the props.conf

[logs-20191210-too_small]
TIME_FORMAT = %H:%M:%S.%3N 

also try this one

DATETIME_CONFIG = CURRENT

2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

Ultra Champion

Where did you deploy that props.conf? It should be on the first full splunk enterprise instance that processes the data (so on a HF or Indexer), not on a universal forwarder. Because if even DATETIME_CONFIG = CURRENT isn't working, then it sounds like that props.conf is not taking effect.

0 Karma

Explorer

Logs and enterprise splunk locate on the single server and I just want to do this config affect to specific log. Not whole logs.

0 Karma

Ultra Champion

What do you mean by that? Are you using some transforms to assign this logs-20191210-too_small sourcetype to these events?

If so: that will not work like this. Only indextime config for the original sourcetype will be applied. You can't override the sourcetype and then apply different indextime config based on that new sourcetype value.

0 Karma

Explorer

There is no transforming here.

0 Karma

SplunkTrust
SplunkTrust

you can use the _time to be the index time in props.conf

 [mysourcetype]
 DATETIME_CONFIG = CURRENT

now you have 2 time fields, _time and your field with HMS.MS

hope it helps

0 Karma

Explorer

should I do something else after this change? e.g. restart splunk service?

0 Karma

SplunkTrust
SplunkTrust

Do you have TIME_FORMAT = %H:%M:%S.%3N in your props.conf?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Explorer

1-Add this props.conf
2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

SplunkTrust
SplunkTrust

What should the correct timestamp be?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Explorer

Hour:Minutes:Seconds:Millisecond
08:00:00.122 = 08 AM

0 Karma