My data files are in Avro, and I have a props.conf that looks like
[source::/logs/...] sourcetype = api [api] KV_MODE = json TIME_PREFIX = "timestamp" TIME_FORMAT = %10s%3N
For a given time range,
How can I resolve this?
In Hunk 6.2 the recommended way of solving this would be to tell Hunk to always return the timestamp field
[my-virtual-index] .... # required fields (6.2 or later) vix.input.[N].required.fields = <comma delimited, optionally wildcarded, list of fields to always output for this input>
In Hunk 6.1.x the recommended way of solving this issue is to disable the column projection optimization that leads to this problem:
[my-provider] ... # disable column projection vix.splunk.search.column.filter = false
The reason for the difference in behavior is due to Hunk's optimization based on required fields - which in "Fast mode" are whatever the search requires, while "Smart mode", which for an event search (e.g. search index=avro) is the same as "Verbose mode", all fields are required. The avro record reader honors the required fields and since Hunk needs/expects "_time" while the data contains "timestamp" the time related field is omitted and thus causing the problem.