Getting Data In

How to configure both CLEAN_KEYS=false in transforms.conf and KV_MODE=auto in props.conf?

Splunk Employee
Splunk Employee

My logs contain many kv pairs, and some field names contain hyphens characters as well:

timestamp="PST 2015-12-01 11:26:36,400", level="INFO",  x-sid="123456789",  x-ip="" ,  x-state="ALIVE"

My goal is to retain the hyphen - characters in the field names; thus setting CLEAN_KEYS = false in transforms.conf, and also like splunk to auto-extract the kv pairs:

# props.conf
KV_MODE = auto
REPORT-blah = keephyphen

# transforms.conf
CLEAN_KEYS = false

However, despite CLEAN_KEYS = false already set in transforms.conf, splunk still replaces the hyphens in the field names with underscore characters:

x-sid --> x_sid, x-ip --> x_ip , x-state --> x_state

Did I misconfigure the transforms.conf stanza, or does Splunk not support CLEAN_KEYS=false and KV_MODE=auto combination?

0 Karma

Splunk Employee
Splunk Employee

Field names must start with a letter and contain only letters, numbers, and underscores. Through testing this it looks like Splunk will not retain (by using CLEAN_KEYS = false) any invalid characters for the field name such as - or / (I see that field names that contain a / are dropped completely where fields containing hyphens are changed to underscore. There is an outstanding enhancement request open (SPL-111920) for such behavior for KV_MODE=auto and CLEAN_KEYS=false to retain the hyphen in the field name. This limitation I'm guessing is because the regex engine doesn't consider those characters to be a valid group structure for the named capture group.

0 Karma

Esteemed Legend

MY suspicion is that the mytest specification in props.conf is NOT being triggered at all and that KV_MODE=auto and CLEAN_KEYS=false are working because those are the default values. So make sure that your stuff has source value of mytest from the get-go (not after being overridden) and this should work.

0 Karma


It looks ok to me - have you run $SPLUNK_HOME/bin/splunk cmd btool transforms list --debug and $SPLUNK_HOME/bin/splunk cmd btool props list --debug to make sure that the stanzas are being read correctly?

0 Karma


Just a quick question, in props, don't you need to mention a spec instead of mytest like source,sourcetype or host just to tell splunk where to apply the transform?

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

.conf23 | Get Your Cybersecurity Defense Analyst Certification in Vegas

We’re excited to announce a new Splunk certification exam being released at .conf23! If you’re going to Las ...

Starting With Observability: OpenTelemetry Best Practices

Tech Talk Starting With Observability: OpenTelemetry Best Practices Tuesday, October 17, 2023   |  11AM PST / ...

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW! Every day the list of sources Admins are responsible for gets bigger and bigger, often making ...