With Splunk HEC it is possible to send a HTTP POST with Json payload to services/collector/event. This supports the fields Json key, that enables you to add additional data to an event that is not present in the _raw (or event) data.
Given the following json payload you should be able to search search index=* k8s_node="node01*":
There are additional configuration that is needed to use fields in some cases. The data is automatically indexed (without the need for props and transformations on the HF/peer), but to be able to utilize the data configuration is needed at the SHC.
With a configuration that is incomplete it will be possible to view the field while searching, but not possible to search with the field specified. The following steps helped resolve the issue:
First check that the events actually are indexed: |tstast count where index=* k8s_node="node01.domain.tld". If count > 0 the field is indexed (and you have access to it). search index=* k8s_node="node01.domain.tld" would still show 0 events.
Then on the searchhead cluster you need to specify that the field you are looking for is an indexed field. This is done in fields.conf.
INDEXED = true
If you are using a shdeployer (or another app on the searchhead for configuration rather than system) you must ensure that metadata/default.meta contains the following.
export = system
Additional search time extraction will need to be specified in props.conf with settings like KV_MODE=auto depending on your source/sourcetypes