Getting Data In
Highlighted

After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

New Member

I am trying to set up KVMODE = autoescaped for a particular source. The stanza looks like the following:

[source:///var/log/test.log]
KV_MODE = auto_escaped

I used the test data directly from the Splunk documentation:
field = "value with \"nested\" quotes."

The resulting search shows the field, field with a value with \.

I have set this in the props.conf on the deployer in the following areas:

$SPLUNKHOME/etc/master-apps/_cluster/local/props.conf

$SPLUNKHOME/etc/shcluster/apps/search/props.conf

Neither of these produce the correct results.

0 Karma
Highlighted

Re: After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

Legend

The KV_MODE in props.conf only applies to search-time field extractions. This means that automatic field extractions occur only for data that has already been indexed. So in the indexed data, the backslashes must appear. This is actually fairly rare in text files (or data streams).

Can you show an example of some data that results from a search and has escaped characters?

0 Karma
Highlighted

Re: After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

New Member

Here is the data example:

Oct 10 22:53:38 server.com com.MetricsForwarder message_type=raw, TEAM=INGEST, version=2.0, environment=xxx, message="{\"header\":
{\"namespace\":\"xxxx\",\"route-definition-id\":\"mobile-device-id\",\"job-instance-id\":\"123456789\",\"step-name\":\"\",\"step-instance-id\":\"123456789\",\"job-status\":\"SUCCESS\",\"step-status\":\"SUCCESS\",\"message-timestamp\":123456789}

,\"payload\":\"{
n \\\"namespace\\\" : \\\"xx-xxxxx\\\",
n \\\"name\\\" : \\\"xxxx\\\",
n \\\"internal-name\\\" : \\\"xxxx\\\",
n \\\"version\\\" : \\\"1.0\\\",
n \\\"state\\\" : \\\"published\\\",
n \\\"category-internal-name\\\" : \\\"id-mapping\\\",
n \\\"provider-id\\\" : \\\"1234\\\",
n \\\"provider-name\\\" : \\\"XXXX\\\",
n \\\"input\\\" :
{\\n \\\"category-internal-name\\\" : \\\"input\\\",\\n \\\"input-uris\\\" : [ \\\"xx:\/\/xxxxxxxxxx-xxxxx\/prod\/data\/data\/input\/4312\/20161005\/mobile.123456789.DL.send\\\" ]\\n }

,
n \\\"output\\\" : [
{\\n \\\"category-internal-name\\\" : \\\"id-mapping\\\",\\n \\\"output-uris\\\" : [ \\\"nnn:\/\/xxxxx-xxxxx\/prod\/etl\/xxx\/xxxx\/device_processing\/data_canonical\/4312\/xx_xxxxx\/123456789\/123456789\\\" ]\\n }

],
n \\\"update-timestamp\\\" : 123456789,
n \\\"status\\\" : \\\"success\\\",
n \\\"instance-id\\\" : \\\"application_123456789\\\",
n \\\"input-validation\\\" : {
n \\\"threshold\\\" : {
n \\\"exceeds-threshold\\\" : false,
n \\\"exceeds-drop-threshold\\\" : false,
n \\\"exceeds-warn-threshold\\\" : false,
n \\\"percent\\\" :
{\\n \\\"drop-percent\\\" : 100,\\n \\\"warn-percent\\\" : 100\\n }


n },
n \\\"summary\\\" :
{\\n \\\"total-processed\\\" : 95,\\n \\\"total-pass\\\" : 95,\\n \\\"total-drop\\\" : 0,\\n \\\"total-warn\\\" : 0,\\n \\\"total-filter\\\" : 0\\n }

,
n \\\"failed-validations\\\" : [ ]
n },
n \\\"output-validation\\\" : {
n \\\"threshold\\\" : {
n \\\"exceeds-threshold\\\" : false,
n \\\"exceeds-drop-threshold\\\" : false,
n \\\"exceeds-warn-threshold\\\" : false,
n \\\"percent\\\" :
{\\n \\\"drop-percent\\\" : 5,\\n \\\"warn-percent\\\" : 5\\n }


n },
n \\\"summary\\\" :
{\\n \\\"total-processed\\\" : 95,\\n \\\"total-pass\\\" : 95,\\n \\\"total-drop\\\" : 0,\\n \\\"total-warn\\\" : 0,\\n \\\"total-filter\\\" : 0\\n }

,
n \\\"failed-validations\\\" : [ ]
n },
n \\\"output-profile\\\" : {
n \\\"id\\\" : 1,
n \\\"internalName\\\" : \\\"xxxxxxxxxx\\\",
n \\\"name\\\" : \\\"xxxx XXXX xxxx xxxxx\\\",
n \\\"profileDefintionUri\\\" : null,
n \\\"dataSetUri\\\" : null,
n \\\"schema\\\" : null,
n \\\"variables\\\" : null,
n \\\"metrics\\\" : [ {
n \\\"id\\\" : 2,
n \\\"displaySequence\\\" : 2,
n \\\"internalName\\\" : \\\"xxxx_x_x_xxxx_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic comment\\\",
n \\\"metrics\\\" : [ {
n \\\"dimensions\\\" :
{\\n \\\"device_id_type\\\" : \\\"1\\\"\\n }

,
n \\\"value\\\" : 92
n } ]
n }, {
n \\\"id\\\" : 1,
n \\\"displaySequence\\\" : 1,
n \\\"internalName\\\" : \\\"xxxx_x_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic field\\\",
n \\\"metrics\\\" : [
{\\n \\\"dimensions\\\" : null,\\n \\\"value\\\" : 92\\n }

]
n }, {
n \\\"id\\\" : 1,
n \\\"displaySequence\\\" : 0,
n \\\"internalName\\\" : \\\"xxxx_x_xx_xxxx\\\",
n \\\"name\\\" : \\\"Generic Field 2\\\",
n \\\"metrics\\\" : [
{\\n \\\"dimensions\\\" : null,\\n \\\"value\\\" : 95\\n }

]
n } ]
n },
n \\\"process-counts\\\" :
{\\n \\\"input\\\" : 95,\\n \\\"input-validation-input\\\" : 95,\\n \\\"input-validation-output\\\" : 95,\\n \\\"transform-input\\\" : 95,\\n \\\"transform-output\\\" : 95,\\n \\\"output-validation-input\\\" : 95,\\n \\\"output-validation-output\\\" : 95,\\n \\\"output-id-mapping\\\" : 95\\n }

\\n}\"}"
0 Karma
Highlighted

Re: After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

New Member

Did the above example help to show the issue we are having? Hoping to get more clarification on this issue.

0 Karma
Highlighted

Re: After deploying KV_MODE = auto_escaped in props.conf to my search head cluster, why are we seeing unexpected search results?

New Member

Wanted to ask one last time, did the above example help to show the data we are collecting and if it will be split right with the auto kv_mode?

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.