topic Re: Performance difference between using SEDCMD and older REGEX/TRANSFORMS method in Monitoring Splunk

Performance difference between using SEDCMD and older REGEX/TRANSFORMS method

DrewO — Wed, 01 Dec 2010 14:50:20 GMT

Is there a performance difference between using the SEDCMD syntax in just props.conf versus using the older method which uses TRANSFORMS and calls a stanza in transforms.conf using REGEX?

Re: Performance difference between using SEDCMD and older REGEX/TRANSFORMS method

Lowell — Wed, 01 Dec 2010 22:26:56 GMT

Just to be clear, the SEDCMD and TRANSFORMS index-time transformations are not exact drop-in replacements for each other. For example, SEDCMD character substitution (like y/ABC/abc/), and repeating replacements (like s/eggs/spam/g) are things that can only be done using SEDCMD, but SEDCMD is limited to only modifying the _raw field. So in your question, I'm assuming that your are specifically asking about where these two approaches overlap (a single replace operation on the _raw field.) If this is incorrect, please update your question.

Re: Performance difference between using SEDCMD and older REGEX/TRANSFORMS method

DrewO — Thu, 02 Dec 2010 08:54:22 GMT

@Lowell yes that's what the student wanted to know. For one off replacements, like overwriting a credit card number/account code.

Re: Performance difference between using SEDCMD and older REGEX/TRANSFORMS method

gkanapathy — Thu, 02 Dec 2010 15:47:53 GMT

For the particular case of s/one/two/ vs ^(.*?)one(.*)$ -> $1two$2, the former is possibly very slightly faster, but differences in the efficiencies of the regex used (e.g., using (.*?) vs (.*)) would probably enormously outweigh that. Note that it's actually not possible to use s/one/two/g (i.e., with the g flag) to replace multiple occurrences with TRANSFORMS.