Out of the box, the unix
sed command operates on a line-by-line basis. Is this the same for the
SEDCMD setting in
props.conf? Or does it operate on the entire event?
Just for the sake of an example. Say I'm using the following unix command to filter out comment lines (lines starting with a ";"). (Technically, this example only clears the comment lines rather than remove them, "grep" would be more obvious choice of unix commands, but splunk doesn't have a
GREPCMD, so it's a mute point.)
cat myfile | sed -re 's/^\s*;.*$//'
Will that work the same way as the following
SEDCMD-drop_comments = s/^\*;.*$//
To get the
SEDCMD to work on a line-by-line basis, you can use the following:
SEDCMD-drop_comments = s/(?m)^\*;.*$//g
The difference is:
(?m)enables multi-line mode in regex engine. This causes
$to now mean start-of-line and end-of-line, rather than start-of-event and end-of-event.
gon the end tells the sed command to be applied as many times as possible; therefore the replacement will occur more than once. So without the
g, only the first matching line would be replaced instead of all the lines.
@gkanapathy, very good point. (Yeah, strait sed-style regex are a pain once you've gotten use to a powerful regex engine. I normally end up using the
sed -r which enables extended regexs, which are still pretty weak when compared to PCRE.) Splunk is definitely more flexible in that regard 😉