Getting Data In

Does SEDCMD work line-by-line or on the entire event?

Lowell
Super Champion

Out of the box, the unix sed command operates on a line-by-line basis. Is this the same for the SEDCMD setting in props.conf? Or does it operate on the entire event?


Just for the sake of an example. Say I'm using the following unix command to filter out comment lines (lines starting with a ";"). (Technically, this example only clears the comment lines rather than remove them, "grep" would be more obvious choice of unix commands, but splunk doesn't have a GREPCMD, so it's a mute point.)

cat myfile |  sed -re 's/^\s*;.*$//'

Will that work the same way as the following props.conf entry:

SEDCMD-drop_comments = s/^\*;.*$//
Tags (1)
1 Solution

Ledion_Bitincka
Splunk Employee
Splunk Employee

splunk's SEDCMD works with one event at a time

View solution in original post

Lowell
Super Champion

To get the SEDCMD to work on a line-by-line basis, you can use the following:

SEDCMD-drop_comments = s/(?m)^\*;.*$//g

The difference is:

  1. The (?m) enables multi-line mode in regex engine. This causes ^ and $ to now mean start-of-line and end-of-line, rather than start-of-event and end-of-event.
  2. The g on the end tells the sed command to be applied as many times as possible; therefore the replacement will occur more than once. So without the g, only the first matching line would be replaced instead of all the lines.

Lowell
Super Champion

@gkanapathy, very good point. (Yeah, strait sed-style regex are a pain once you've gotten use to a powerful regex engine. I normally end up using the sed -r which enables extended regexs, which are still pretty weak when compared to PCRE.) Splunk is definitely more flexible in that regard 😉

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Note that Splunk's SEDCMD uses PCRE regex, not standard sed regex.

0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

splunk's SEDCMD works with one event at a time

Lowell
Super Champion

That's what I was suspecting. I found you can make it work line-by-line by using multi-line regex mode. (I added an answer of my own with an example.)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...