Getting Data In

How to configure both CLEAN_KEYS=false in transforms.conf and KV_MODE=auto in props.conf?

Splunk Employee
Splunk Employee

My logs contain many kv pairs, and some field names contain hyphens characters as well:

timestamp="PST 2015-12-01 11:26:36,400", level="INFO",  x-sid="123456789",  x-ip="" ,  x-state="ALIVE"

My goal is to retain the hyphen - characters in the field names; thus setting CLEAN_KEYS = false in transforms.conf, and also like splunk to auto-extract the kv pairs:

# props.conf
KV_MODE = auto
REPORT-blah = keephyphen

# transforms.conf
CLEAN_KEYS = false

However, despite CLEAN_KEYS = false already set in transforms.conf, splunk still replaces the hyphens in the field names with underscore characters:

x-sid --> x_sid, x-ip --> x_ip , x-state --> x_state

Did I misconfigure the transforms.conf stanza, or does Splunk not support CLEAN_KEYS=false and KV_MODE=auto combination?

0 Karma

Splunk Employee
Splunk Employee

Field names must start with a letter and contain only letters, numbers, and underscores. Through testing this it looks like Splunk will not retain (by using CLEAN_KEYS = false) any invalid characters for the field name such as - or / (I see that field names that contain a / are dropped completely where fields containing hyphens are changed to underscore. There is an outstanding enhancement request open (SPL-111920) for such behavior for KV_MODE=auto and CLEAN_KEYS=false to retain the hyphen in the field name. This limitation I'm guessing is because the regex engine doesn't consider those characters to be a valid group structure for the named capture group.

0 Karma

Esteemed Legend

MY suspicion is that the mytest specification in props.conf is NOT being triggered at all and that KV_MODE=auto and CLEAN_KEYS=false are working because those are the default values. So make sure that your stuff has source value of mytest from the get-go (not after being overridden) and this should work.

0 Karma


It looks ok to me - have you run $SPLUNK_HOME/bin/splunk cmd btool transforms list --debug and $SPLUNK_HOME/bin/splunk cmd btool props list --debug to make sure that the stanzas are being read correctly?

0 Karma


Just a quick question, in props, don't you need to mention a spec instead of mytest like source,sourcetype or host just to tell splunk where to apply the transform?

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

Admin Your Splunk Cloud, Your Way

Join us to maximize different techniques to best tune Splunk Cloud. In this Tech Enablement, you will get ...

Cloud Platform | Discontinuing support for TLS version 1.0 and 1.1

Overview Transport Layer Security (TLS) is a security communications protocol that lets two computers, ...

New Customer Testimonials

Enterprises of all sizes and across different industries are accelerating cloud adoption by migrating ...