Interesting fields disappear as number of events returned is increased

[alanzchan]

For example, here is data from the last 60 minutes. Less events are returned and the index, source, and sourcetype fields are still there.

However, when I search for events from the last 24 hours, the index, source, and sourcetype fields dissappears.

I'm already searching in verbose mode and looking at all fields as well so this is not the issue. I think there is some limits configuration that I should set to prevent this from happening. Has anyone experienced this or have any suggestions?

[ajjain]

@alanzchan  Were you able to find a solution to your problem ?

[woodcock]

The only way for the fields to disappear is if something made them disappear (like a calculated field).

[alanzchan]

Calculated fields are fields added to events at search time. I'm not doing any calculated fields. Events are coming in via HEC with index already defined in the outputs of the fluentd agent.

[woodcock]

Click on the All Fields link and make sure that the fields that you care about are selected (or select them). There is a concept called viewstate that controls this but the gist is that the selected fields are more-or-less supposed to follow you around, provided the fields exist (and the ones that you listed should always exist).

[alanzchan]

I don't think this is the issue. I'm already searching in verbose mode and I've looked in All Fields (including setting the coverage to All Fields). The fields just disappear. The indexes are still searchable, but it just doesn't show up in the list.