Getting Data In

How to prevent Splunk from converting time to date

indeed_2000
Builder

Hi,
I have a log file like this:

08:00:00.032 user     parameter: A[0]B[0]C: Action successful.

This is just hour:minutes:seconds:mile seconds
As you see the only time of that event exists in log, and Splunk _time automatically converts this 08:00:00.032 to timestamp!
This is the cause of the wrong date for events.

For example, If I added today's log file to Splunk it will show events belong to 2018 ,2017, 2016 ...

Any recommendation?
Thanks.

0 Karma

FrankVl
Ultra Champion

It is not very clear what your issue is here. Since the timestamp does not include a date, I believe Splunk will assume the date is today, and index all events as such.

Do you have logs from multiple dates in your file, but without a date in the events? Or what exactly is your issue? Maybe show what the _time field looks like in Splunk and explain what is wrong with it.

0 Karma

indeed_2000
Builder

1-file contain only log of today.
2-there is no date in log it just time 08:00:00.122
3-"_time" also show logs that belong this dates 2018 ,2017, 2016 ...

exact problem is splunk convert time to date e.g.

source="/opt/logs-20191210.log" | table _time

1   2018-02-02 09:04:04.042
2   2018-02-02 09:04:04.041
3   2018-02-02 09:04:04.041
4   2018-02-02 09:04:04.039
5   2018-02-02 09:04:04.039

....

1   2017-07-13 08:43:56.928
2   2017-07-13 08:43:56.927
3   2017-07-13 08:43:56.925
4   2017-07-13 08:43:56.925
5   2017-07-13 08:43:56.920

....

1   2016-12-26 08:48:35.986
2   2016-12-26 08:48:35.986
3   2016-12-26 08:48:35.984
4   2016-12-26 08:48:35.979
5   2016-12-26 08:48:35.979

...

1-here is the props.conf

[logs-20191210-too_small]
TIME_FORMAT = %H:%M:%S.%3N 

also try this one

DATETIME_CONFIG = CURRENT

2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

FrankVl
Ultra Champion

Where did you deploy that props.conf? It should be on the first full splunk enterprise instance that processes the data (so on a HF or Indexer), not on a universal forwarder. Because if even DATETIME_CONFIG = CURRENT isn't working, then it sounds like that props.conf is not taking effect.

0 Karma

indeed_2000
Builder

Logs and enterprise splunk locate on the single server and I just want to do this config affect to specific log. Not whole logs.

0 Karma

FrankVl
Ultra Champion

What do you mean by that? Are you using some transforms to assign this logs-20191210-too_small sourcetype to these events?

If so: that will not work like this. Only indextime config for the original sourcetype will be applied. You can't override the sourcetype and then apply different indextime config based on that new sourcetype value.

0 Karma

indeed_2000
Builder

There is no transforming here.

0 Karma

adonio
SplunkTrust
SplunkTrust

you can use the _time to be the index time in props.conf

 [mysourcetype]
 DATETIME_CONFIG = CURRENT

now you have 2 time fields, _time and your field with HMS.MS

hope it helps

0 Karma

indeed_2000
Builder

should I do something else after this change? e.g. restart splunk service?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Do you have TIME_FORMAT = %H:%M:%S.%3N in your props.conf?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

indeed_2000
Builder

1-Add this props.conf
2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What should the correct timestamp be?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

indeed_2000
Builder

Hour:Minutes:Seconds:Millisecond
08:00:00.122 = 08 AM

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!