topic Re: Accelerated data model not storing milliseconds in _time in Reporting

Accelerated data model not storing milliseconds in _time

payal23 — Wed, 30 Sep 2020 01:39:46 GMT

I am not able to capture milliseconds in accelerated data model.

Query is like :

|tstats max(_time) as Start min(_time) as End from datamodel=One.A where nodename=B.A by A.Id|eval duration=End-Start

Raw logs are having milliseconds but the above query is not having the milliseconds.

Thanks
PAyal

Re: Accelerated data model not storing milliseconds in _time

jawaharas — Thu, 08 Aug 2019 07:06:45 GMT

May I know the name of datamodel you are using?

Ideally below query return _time in epoch format. What you get?

|tstats max(_time) as Start min(_time) as End from datamodel=One

Re: Accelerated data model not storing milliseconds in _time

payal23 — Thu, 08 Aug 2019 08:00:32 GMT

yes. In epoch format. But that value does not have milliseconds.

So, in raw logs time is 5:49:08.715 PM and the tstats epoch converted time has 1565250548.

Re: Accelerated data model not storing milliseconds in _time

jawaharas — Fri, 09 Aug 2019 03:39:16 GMT

I hope you can see the milliseconds when you run below query.

| from datamodel One
| table _time

Re: Accelerated data model not storing milliseconds in _time

payal23 — Thu, 22 Aug 2019 05:27:10 GMT

yes I am able to see here.. but if i do max or min of the time.. millisecond is not printing

Re: Accelerated data model not storing milliseconds in _time

jawaharas — Mon, 26 Aug 2019 04:59:44 GMT

I can't reproduce the issue with below query on 'Authentication' datamodel.

|tstats max(_time) as Start min(_time) as End from datamodel=One

Is yours custom datamodel? if it's standard one, can you share the datamodel name?

Re: Accelerated data model not storing milliseconds in _time

Sukisen1981 — Mon, 26 Aug 2019 07:12:26 GMT

hi @payal23
What happens if you try this

|tstats max(_time) as Start min(_time) as End from datamodel=One.A where nodename=B.A by A.Id| eval newstrt=strftime(Start,"%Y-%m-%d %H:%M:%S:%3N %p")| eval newend=strftime(End,"%Y-%m-%d %H:%M:%S:%3N %p")

Can you see milliseconds now?

Re: Accelerated data model not storing milliseconds in _time

payal23 — Tue, 03 Sep 2019 02:37:59 GMT

Hi Suki, @Sukisen1981

if i search for last 15 - 20 mins milliseconds are displaying.. But if i search any older time than this.. then it is not displaying milliseconds.

My question is that while saving the data in the indexer (accelerated data model) will it not save the milliseconds?

Re: Accelerated data model not storing milliseconds in _time

payal23 — Tue, 03 Sep 2019 02:39:46 GMT

@jawaharas It's a custom data model. Can you try accelerating that data model and look for milliseconds?

Re: Accelerated data model not storing milliseconds in _time

jawaharas — Tue, 03 Sep 2019 05:30:26 GMT

We can see milliseconds in accelerated data model.

Re: Accelerated data model not storing milliseconds in _time

BernardEAI — Wed, 28 Apr 2021 12:24:17 GMT

I am experiencing the same problem. When searching an accelerated data model, the miliseconds of the _time field is lost (when using a tstats latest search). This is a problem, since the miliseconds are needed to accurately determine the latest event!

I describe the problem in more detail in a question I posted here:

https://community.splunk.com/t5/Splunk-Search/Data-model-time-field-format/m-p/549121#M155777