Getting Data In

How to prevent Splunk from converting time to date

indeed_2000
Motivator

Hi,
I have a log file like this:

08:00:00.032 user     parameter: A[0]B[0]C: Action successful.

This is just hour:minutes:seconds:mile seconds
As you see the only time of that event exists in log, and Splunk _time automatically converts this 08:00:00.032 to timestamp!
This is the cause of the wrong date for events.

For example, If I added today's log file to Splunk it will show events belong to 2018 ,2017, 2016 ...

Any recommendation?
Thanks.

0 Karma

FrankVl
Ultra Champion

It is not very clear what your issue is here. Since the timestamp does not include a date, I believe Splunk will assume the date is today, and index all events as such.

Do you have logs from multiple dates in your file, but without a date in the events? Or what exactly is your issue? Maybe show what the _time field looks like in Splunk and explain what is wrong with it.

0 Karma

indeed_2000
Motivator

1-file contain only log of today.
2-there is no date in log it just time 08:00:00.122
3-"_time" also show logs that belong this dates 2018 ,2017, 2016 ...

exact problem is splunk convert time to date e.g.

source="/opt/logs-20191210.log" | table _time

1   2018-02-02 09:04:04.042
2   2018-02-02 09:04:04.041
3   2018-02-02 09:04:04.041
4   2018-02-02 09:04:04.039
5   2018-02-02 09:04:04.039

....

1   2017-07-13 08:43:56.928
2   2017-07-13 08:43:56.927
3   2017-07-13 08:43:56.925
4   2017-07-13 08:43:56.925
5   2017-07-13 08:43:56.920

....

1   2016-12-26 08:48:35.986
2   2016-12-26 08:48:35.986
3   2016-12-26 08:48:35.984
4   2016-12-26 08:48:35.979
5   2016-12-26 08:48:35.979

...

1-here is the props.conf

[logs-20191210-too_small]
TIME_FORMAT = %H:%M:%S.%3N 

also try this one

DATETIME_CONFIG = CURRENT

2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

FrankVl
Ultra Champion

Where did you deploy that props.conf? It should be on the first full splunk enterprise instance that processes the data (so on a HF or Indexer), not on a universal forwarder. Because if even DATETIME_CONFIG = CURRENT isn't working, then it sounds like that props.conf is not taking effect.

0 Karma

indeed_2000
Motivator

Logs and enterprise splunk locate on the single server and I just want to do this config affect to specific log. Not whole logs.

0 Karma

FrankVl
Ultra Champion

What do you mean by that? Are you using some transforms to assign this logs-20191210-too_small sourcetype to these events?

If so: that will not work like this. Only indextime config for the original sourcetype will be applied. You can't override the sourcetype and then apply different indextime config based on that new sourcetype value.

0 Karma

indeed_2000
Motivator

There is no transforming here.

0 Karma

adonio
Ultra Champion

you can use the _time to be the index time in props.conf

 [mysourcetype]
 DATETIME_CONFIG = CURRENT

now you have 2 time fields, _time and your field with HMS.MS

hope it helps

0 Karma

indeed_2000
Motivator

should I do something else after this change? e.g. restart splunk service?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Do you have TIME_FORMAT = %H:%M:%S.%3N in your props.conf?

---
If this reply helps you, Karma would be appreciated.
0 Karma

indeed_2000
Motivator

1-Add this props.conf
2- remove fishbucket
3- restart service

Problem still remain.

Any idea?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What should the correct timestamp be?

---
If this reply helps you, Karma would be appreciated.
0 Karma

indeed_2000
Motivator

Hour:Minutes:Seconds:Millisecond
08:00:00.122 = 08 AM

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...