collect command/sourcetypes and \ character

bowesmana — Wed, 10 Mar 2021 06:45:22 GMT

Came across an interesting behaviour with collect today depending on whether you specify a sourcetype or not. If you have a field containing a \ character it will escape the \ when using a sourcetype, but not with stash.

These two searches

| makeresults | eval field="App\X" | collect index=main sourcetype="something_other_than_stash"

| makeresults | eval field="App\X" | collect index=main

will generate two different field values in index for 'field'

When using the first with a sourcetype, the resultant field has two \\ characters in the field value in the index.

Both examples show the raw event as App\\X, but fieldsummary shows the one including sourcetype to be App\\\\X

Anyone know why this is?

Re: collect command/sourcetypes and \ character

bowesmana — Wed, 10 Mar 2021 07:15:03 GMT

OK, so the answer is related to KV_MODE.

With stash, default KV_MODE is none, hence no escaping of the extracted fields is done. With a specified sourcetype the default will be KV_MODE=auto, so the data will be escaped when extracted.

Thanks to firebus on the apac slack channel

Also using KV_MODE=auto_escaped works and that is specifically documented to honour escaped sequences within quoted strings

topic collect command/sourcetypes and \ character in Splunk Search

collect command/sourcetypes and \ character

Re: collect command/sourcetypes and \ character