My logs contain many kv pairs, and some field names contain hyphens characters as well:
timestamp="PST 2015-12-01 11:26:36,400", level="INFO", x-sid="123456789", x-ip="127.0.0.1" , x-state="ALIVE"
My goal is to retain the hyphen
- characters in the field names; thus setting
CLEAN_KEYS = false in transforms.conf, and also like splunk to auto-extract the kv pairs:
# props.conf [mytest] NO_BINARY_CHECK = true KV_MODE = auto REPORT-blah = keephyphen # transforms.conf [keephyphen] CLEAN_KEYS = false
CLEAN_KEYS = false already set in transforms.conf, splunk still replaces the hyphens in the field names with underscore characters:
x-sid --> xsid, x-ip --> xip , x-state --> x_state
Did I misconfigure the transforms.conf stanza, or does Splunk not support CLEANKEYS=false and KVMODE=auto combination?
Just a quick question, in props, don't you need to mention a spec instead of mytest like source,sourcetype or host just to tell splunk where to apply the transform?
It looks ok to me - have you run
$SPLUNK_HOME/bin/splunk cmd btool transforms list --debug and
$SPLUNK_HOME/bin/splunk cmd btool props list --debug to make sure that the stanzas are being read correctly?
MY suspicion is that the
mytest specification in
props.conf is NOT being triggered at all and that
CLEAN_KEYS=false are working because those are the default values. So make sure that your stuff has
source value of
mytest from the get-go (not after being overridden) and this should work.
Field names must start with a letter and contain only letters, numbers, and underscores. Through testing this it looks like Splunk will not retain (by using CLEANKEYS = false) any invalid characters for the field name such as - or / (I see that field names that contain a / are dropped completely where fields containing hyphens are changed to underscore. There is an outstanding enhancement request open (SPL-111920) for such behavior for KVMODE=auto and CLEAN_KEYS=false to retain the hyphen in the field name. This limitation I'm guessing is because the regex engine doesn't consider those characters to be a valid group structure for the named capture group.