FORMAT = <string> * The special identifier $0 represents what was in the DEST_KEY before this regex was performed.
Can I use $0 in DEFAULT_VALUE, too ? My intention is to add a default value to the current content of the source key, if the regex doesn't match.
Any help appreciated.
Here's some more information about what I want to do. I need to build the index name from several fields in the record (e.g. stage and logtype). In some cases I can directly use the value from the record in some other cases not. I tried the following:
props.conf: TRANSFORMS-index_stage = index_stage_prod, index_stage_test, index_logtype ... transforms.conf: [index_stage_prod] DEST_KEY = _MetaData:Index REGEX = (?i)stage=(PROD|BCP).* FORMAT = "idx_prod" [index_stage_test] DEST_KEY = _MetaData:Index REGEX = (?i)stage=(DEV|TEST).* FORMAT = "idx_test" [index_logtype] DEST_KEY = _MetaData:Index REGEX = (?i)logtype=([^\s\t\r]*).* FORMAT = $0_$1 DEFAULT_VALUE = $0_unclassified
The seem to be at least two problems: 1) It seems that building the index name incrementally doesn't work. My tests showed that if the first transform was successful the other transforms do not fire. 2) I need "$0" to work in DEFAULT_VALUE
I workaround would be to match all values with one regular expression, but that has some limitations. I will also open a case about that problem.
I don't think this is possible. In the common field extraction (fields loaded at search-time) it is not possible to augment the regex extracted values. So it seems unlikely that you could augment the previous value of the
$0) with some additional static text that would work when you don't have a match. Certainly there would be ways of accomplishing this using an
eval statement, but I don't think you can do this simply with a transformer.
Here is an example demonstrating what I mean about adding static text to a field. This example will not work:
[my-bogus-transformer] REGEX = \s(\d+)\s FORMAT = my_field::"$1 (int)"
This does not work, because the text " (int)" is NOT part of the raw text and you can't just arbitrarily add text to extracted fields like this. (Unless you use do index-time field extractions, which I don't recommend here.) My understanding is that this has to do with search performance and the fact that a search like
field1=joe is turned into the search
joe AND field1="joe".
Hmm, that's a very different scenario that I thought you were asking about. Index-time transforms don't have the limitation that I described above, but I'm not sure knowing that solves anything.... I think if you update your questions with a use-case example or two, there may be another possible solution. It's also possible this should be a feature request, but either way you'll need to give more details about what your use-case is.
Wollinet, you are correct. I've updated by answer and removed my incorrect understanding of how splunk is using
$0 in this case vs the traditional regex meaning of
$0. I think an
eval based approach is your best (and possibly only) option.
$0 is used differently here. The documentation is correct as $0 is original value of DEST_KEY. I've already tested that.
What I want is to add something to DEST_KEY (which already contains a value) if the regex matches and a default value if not. The problem is that the default value should be added to DEST_KEY and not overwrite it.