Trying to properly perform an ingestion time SED

Cornisgud — Sun, 25 Feb 2024 19:57:10 GMT

Hello,

Currently I'm attempting to make a CommandHistory field a bit more readable for our analysts but I'm having trouble getting the formatting correct or maybe I'm just using the wrong command or taking the wrong approach.

Basically our EDR dumps recent commands ran on a system into the CommandHistory field separated by a ¶ symbol. I'm trying to just replace that with a new line at ingestion time.

Made up example of what's in CommandHistory at the moment (I don't want to use real data I apologize):
command1 -q lifeishard¶ReallyLong Command -t LifeIsHarderWhenYouCantFigureItOut¶ThirdCommand -u switchesare -cool¶One more command

The search time commands that get me what I want in a field called commandHistory_sed:
| eval commandHistory = CommandHistory
| rex field=commandHistory_sed mode=sed "s/\¶/\n/g"

This ends up looking like this:
command1 -q lifeishard
ReallyLong Command -t LifeIsHarderWhenYouCantFigureItOut
ThirdCommand -u switchesare -cool
One more command

What I've tried in props.conf:
SEDCMD-substitute = 's/\¶/\n/g'
SEDCMD-alter = 's/\¶/\n/g'

Neither work. We have many other Eval and FIELDALIAS statements under this sourcetype in props.conf that are functioning fine so I think I'm just not formatting the SED properly or I'm not taking the right approach.

Does anyone have any advice on what I am doing wrong and what I need to do to achieve the result?

Thank you for any help in advance!

Re: Trying to properly perform an ingestion time SED

richgalloway — Sun, 25 Feb 2024 20:35:59 GMT

The rex command needs the name of an existing field in the field option. Try this

| eval commandHistory = CommandHistory | rex field=commandHistory mode=sed "s/\¶/\n/g"

Re: Trying to properly perform an ingestion time SED

Cornisgud — Sun, 25 Feb 2024 20:41:10 GMT

Sorry, I made a typo in the search time that gets me what I need it was supposed to say:

| eval CommandHistory = commandHistory_sed

I can make the effect happen in search time, the issue is I need to figure out how to have this effect applied at ingest time so the effect is automatically applied to all of the events.

topic Re: Trying to properly perform an ingestion time SED in Getting Data In

Trying to properly perform an ingestion time SED

Re: Trying to properly perform an ingestion time SED

Re: Trying to properly perform an ingestion time SED