After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

[dankulinski]

I am trying to set up KV_MODE = auto_escaped for a particular source. The stanza looks like the following:

[source:///var/log/test.log]
KV_MODE = auto_escaped

I used the test data directly from the Splunk documentation:
field = "value with \"nested\" quotes."

The resulting search shows the field, field with a value with \.

I have set this in the props.conf on the deployer in the following areas:

$SPLUNKHOME/etc/master-apps/_cluster/local/props.conf

$SPLUNKHOME/etc/shcluster/apps/search/props.conf

Neither of these produce the correct results.

[lguinn2]

The KV_MODE in props.conf only applies to search-time field extractions. This means that automatic field extractions occur only for data that has already been indexed. So in the indexed data, the backslashes must appear. This is actually fairly rare in text files (or data streams).

Can you show an example of some data that results from a search and has escaped characters?

[dankulinski]

Here is the data example:

Oct 10 22:53:38 server.com com.MetricsForwarder message_type=raw, TEAM=INGEST, version=2.0, environment=xxx, message="{\"header\":
{\"namespace\":\"xxxx\",\"route-definition-id\":\"mobile-device-id\",\"job-instance-id\":\"123456789\",\"step-name\":\"\",\"step-instance-id\":\"123456789\",\"job-status\":\"SUCCESS\",\"step-status\":\"SUCCESS\",\"message-timestamp\":123456789}

,\"payload\":\"{
n \\\"namespace\\\" : \\\"xx-xxxxx\\\",
n \\\"name\\\" : \\\"xxxx\\\",
n \\\"internal-name\\\" : \\\"xxxx\\\",
n \\\"version\\\" : \\\"1.0\\\",
n \\\"state\\\" : \\\"published\\\",
n \\\"category-internal-name\\\" : \\\"id-mapping\\\",
n \\\"provider-id\\\" : \\\"1234\\\",
n \\\"provider-name\\\" : \\\"XXXX\\\",
n \\\"input\\\" :
{\\n \\\"category-internal-name\\\" : \\\"input\\\",\\n \\\"input-uris\\\" : [ \\\"xx:\/\/xxxxxxxxxx-xxxxx\/prod\/data\/data\/input\/4312\/20161005\/mobile.123456789.DL.send\\\" ]\\n }

,
n \\\"output\\\" : [
{\\n \\\"category-internal-name\\\" : \\\"id-mapping\\\",\\n \\\"output-uris\\\" : [ \\\"nnn:\/\/xxxxx-xxxxx\/prod\/etl\/xxx\/xxxx\/device_processing\/data_canonical\/4312\/xx_xxxxx\/123456789\/123456789\\\" ]\\n }

],
n \\\"update-timestamp\\\" : 123456789,
n \\\"status\\\" : \\\"success\\\",
n \\\"instance-id\\\" : \\\"application_123456789\\\",
n \\\"input-validation\\\" : {
n \\\"threshold\\\" : {
n \\\"exceeds-threshold\\\" : false,
n \\\"exceeds-drop-threshold\\\" : false,
n \\\"exceeds-warn-threshold\\\" : false,
n \\\"percent\\\" :
{\\n \\\"drop-percent\\\" : 100,\\n \\\"warn-percent\\\" : 100\\n }


n },
n \\\"summary\\\" :
{\\n \\\"total-processed\\\" : 95,\\n \\\"total-pass\\\" : 95,\\n \\\"total-drop\\\" : 0,\\n \\\"total-warn\\\" : 0,\\n \\\"total-filter\\\" : 0\\n }

,
n \\\"failed-validations\\\" : [ ]
n },
n \\\"output-validation\\\" : {
n \\\"threshold\\\" : {
n \\\"exceeds-threshold\\\" : false,
n \\\"exceeds-drop-threshold\\\" : false,
n \\\"exceeds-warn-threshold\\\" : false,
n \\\"percent\\\" :
{\\n \\\"drop-percent\\\" : 5,\\n \\\"warn-percent\\\" : 5\\n }


n },
n \\\"summary\\\" :
{\\n \\\"total-processed\\\" : 95,\\n \\\"total-pass\\\" : 95,\\n \\\"total-drop\\\" : 0,\\n \\\"total-warn\\\" : 0,\\n \\\"total-filter\\\" : 0\\n }

,
n \\\"failed-validations\\\" : [ ]
n },
n \\\"output-profile\\\" : {
n \\\"id\\\" : 1,
n \\\"internalName\\\" : \\\"xxxxxxxxxx\\\",
n \\\"name\\\" : \\\"xxxx XXXX xxxx xxxxx\\\",
n \\\"profileDefintionUri\\\" : null,
n \\\"dataSetUri\\\" : null,
n \\\"schema\\\" : null,
n \\\"variables\\\" : null,
n \\\"metrics\\\" : [ {
n \\\"id\\\" : 2,
n \\\"displaySequence\\\" : 2,
n \\\"internalName\\\" : \\\"xxxx_x_x_xxxx_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic comment\\\",
n \\\"metrics\\\" : [ {
n \\\"dimensions\\\" :
{\\n \\\"device_id_type\\\" : \\\"1\\\"\\n }

,
n \\\"value\\\" : 92
n } ]
n }, {
n \\\"id\\\" : 1,
n \\\"displaySequence\\\" : 1,
n \\\"internalName\\\" : \\\"xxxx_x_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic field\\\",
n \\\"metrics\\\" : [
{\\n \\\"dimensions\\\" : null,\\n \\\"value\\\" : 92\\n }

]
n }, {
n \\\"id\\\" : 1,
n \\\"displaySequence\\\" : 0,
n \\\"internalName\\\" : \\\"xxxx_x_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic Field 2\\\",
n \\\"metrics\\\" : [
{\\n \\\"dimensions\\\" : null,\\n \\\"value\\\" : 95\\n }

]
n } ]
n },
n \\\"process-counts\\\" :
{\\n \\\"input\\\" : 95,\\n \\\"input-validation-input\\\" : 95,\\n \\\"input-validation-output\\\" : 95,\\n \\\"transform-input\\\" : 95,\\n \\\"transform-output\\\" : 95,\\n \\\"output-validation-input\\\" : 95,\\n \\\"output-validation-output\\\" : 95,\\n \\\"output-id-mapping\\\" : 95\\n }

\\n}\"}"

[dankulinski]

Wanted to ask one last time, did the above example help to show the data we are collecting and if it will be split right with the auto kv_mode?

[dankulinski]

Did the above example help to show the issue we are having? Hoping to get more clarification on this issue.