How to Filter part of data in an event during index time

[reach2tushar]

Hi, I have a type of following event data which is coming from forwarder:

Column1=XYZ+Column2=ABC+ColumnC=GGG....

I want to remove Column2=ABC value from the event before indexing. Can help how to filter this data. The event should be indexed like this:

Column1=XYZ+ColumnC=GGG....

[diogofgm]

Try this:
props.conf

[yoursource_type]
SEDCMD-removecolumn=s/Column2=[^\+]*\+//g

[reach2tushar]

Hi, thanks for the reply.
I have windows server environment. I tried this but it didn't work.
will SEDCMD only work on Linux server?

[somesoni2]

This should work regardless of OS. Did you put the config in your heavy forwarder/indexers and restarted it? Also, it would be good if you can test the regex against your actual data from www.regex101.com or similar sites before actually using it.

[reach2tushar]

I added these configs on my indexer and restarted. should I move the configs to forwarder?

[somesoni2]

What type of forwarder you've, Universal forwarder OR full Enterprise Instance acting as forwarder? Also, Since you posted dummy data in question, the regex is suggested accordingly. Do remember to validate the regex first (if regex is wrong, the SEDCMD will not work anyways).

[reach2tushar]

I am using full enterprise instance as a forwarder. Also I verified regex with http://www.regexr.com/. It looks good. The SEDCMD is in props.conf file on indexer.

[diogofgm]

Since you are using a heavy forwarder, put the props the forwarder

[reach2tushar]

It worked after moving on heavy forwarder.
Thanks heaps for your help.