Why does a transforms report stanza have issues with source key?

[landen99]

Specifically, I have a search-time transform which works against _raw but not against a source key. When it only specifies a regex and a source key, it does not work, but when the source key is removed, it does work. It would be really good to also gain a general understanding of what contributes to source key not working.

sourcetype: splunkd
regex: \/+handshake\/+reply\/+(?<deploy_client_name>[^\/]+)\/+(?<deploy_client>\S+)
source key: _raw OR channel; the first works and the second does not.
https://regex101.com/r/eI7V1B/2

The field channel extracts normally by default but does not seem to be available for the transform. Is it possible to get the transfer to work against the field, channel?

In any Splunk environment, you should see that channel is extracted by default with the following search:

index=_internal sourcetype=splunkd component=PubSubSvr reply handshake PubSubSvr | dedup channel | table channel deploy*

You can use my regex (above) in a transform on your own search head to duplicate the issue that I am describing. Here is the transform:

[ds_msg_client_handshake]
CLEAN_KEYS = 0
REGEX = \/handshake\/reply\/(?P<deploy_client_name>[^\/]+)\/(?P<deploy_client>\S+)
SOURCE_KEY = channel

[bheemireddi]

Yes, In order to use for the field to be SOURCE_KEY in the REPORT/transforms - you either get that through EXTRACT OR REPORT and make sure the stanza that is extracting the SOURCE_KEY evaluates before the stanza where you want to use it

[landen99]

I think that you meant to make this a reply to my comment on your answer above...

[bheemireddi · Accepted solution]

Yes, all the field extractions for the splunkd source type are through KV_MODE, since this is auto by default. So I am not surprised it did not work as the SOURCE_KEY, as the field extractions also have the precedence of their operations.

REPORT goes first before KV_MODE, and the channel field won't be available for your REPORT in the transforms.conf

For more understanding on the precedence, probably below thread might help
https://answers.splunk.com/answers/475935/for-a-field-user-which-has-precedence-an-eval-defi.html

[landen99]

So is the solution then to configure a props to extract the field first so that it is available before the transform's report call? Or is there a better solution?

[bheemireddi]

how is the channel field getting extracted? using another transforms?KV_MODE? did notice something before it doesn't do much if it is from the KV_MODE.

Can you paste sample transforms where you are using SOURCE_KEY? and also how the channel field getting extracted in this case?

[landen99]

Here is the transform stanza:

[ds_msg_client_handshake]
CLEAN_KEYS = 0
REGEX = \/handshake\/reply\/(?P<deploy_client_name>[^\/]+)\/(?P<deploy_client>\S+)
SOURCE_KEY = channel

[landen99]

The channel field is extracted by default against the splunkd sourcetype. I didn't do anything to extract "channel". You can see the events in your own deployments by searching:

index=_internal sourcetype=splunkd component=PubSubSvr reply handshake PubSubSvr | dedup channel | table channel deploy*

[somesoni2]

This transform is doing index-time activity or search time?

[landen99]

search time