Datamodel missing field extractions, but base search returns those fields accurately.

[cdhippen]

I have a datamodel lets say with a base constraint that returns the following two events

01-01-2019 01:00:00 type=VIEW_REQUEST duration=100 taskID=123456
01-01-2019 00:00:00 request=do_something count=5 taskID=123456

And I have a search built that was returning null results for taskID 123456 type and duration, but was returning the request and count fields, i.e. it was missing the fields from the VIEW_REQUEST event log. I ran the base search for the taskID 123456 and the fields are extracted properly and can be used without issue.

To summarize, this search:

("VIEW_REQUEST" "duration") OR ("do_something" "count") taskID="123456" 
| stats values(type) as type values(request) as request values(duration) as duration values(count) as count by taskID

returns

type --------| request -----| duration | count | taskID
VIEW_REQUEST | do_something | 100 -----| 5 ----| 123456

While this search (with """("VIEW_REQUEST" "duration") OR ("do_something" "count") taskID""" as the constraint for the datamodel):

| tstats values(request.type) as type values(request.request) as request values(request.duration) as duration values(request.count)  as count from datamodel=request by request.taskID

returns

type | request -----| duration | count | taskID
NULL | do_something | NULL ----| 5 ----| 123456

The field extraction is working correctly for approximately 99.7% of all results, but there are 297 out of approximately 80,000 results where duration and type are null for some reason when searched via the data model, but the fields are extracted properly in the events that are returned by the base constraint. How do I fix this?

[cdhippen]

I can't remember why, but I had a span=1s at the end of the tstats that was removing an extremely small subset of events. Removing that didn't seem to have an effect on the end results that I was expecting other than correctly reporting the durations I was expecting. I believe this is resolved although I'm unsure why it was failing with the span in there.

[cdhippen]

I remember now why I had the span in there, without the span=1s, the span was defaulting to daily. I need to have it on a transactional level

[cdhippen]

Which I think I've now resolved by adding latest(_time) to the aggregations rather than the by clause. I'll update if I find other issues with it.