Getting Data In

How to Filter part of data in an event during index time

reach2tushar
Explorer

Hi, I have a type of following event data which is coming from forwarder:

Column1=XYZ+Column2=ABC+ColumnC=GGG....

I want to remove Column2=ABC value from the event before indexing. Can help how to filter this data. The event should be indexed like this:

Column1=XYZ+ColumnC=GGG....
0 Karma

diogofgm
SplunkTrust
SplunkTrust

Try this:
props.conf

[yoursource_type]
SEDCMD-removecolumn=s/Column2=[^\+]*\+//g
------------
Hope I was able to help you. If so, some karma would be appreciated.

reach2tushar
Explorer

Hi, thanks for the reply.
I have windows server environment. I tried this but it didn't work.
will SEDCMD only work on Linux server?

0 Karma

somesoni2
Revered Legend

This should work regardless of OS. Did you put the config in your heavy forwarder/indexers and restarted it? Also, it would be good if you can test the regex against your actual data from www.regex101.com or similar sites before actually using it.

0 Karma

reach2tushar
Explorer

I added these configs on my indexer and restarted. should I move the configs to forwarder?

0 Karma

somesoni2
Revered Legend

What type of forwarder you've, Universal forwarder OR full Enterprise Instance acting as forwarder? Also, Since you posted dummy data in question, the regex is suggested accordingly. Do remember to validate the regex first (if regex is wrong, the SEDCMD will not work anyways).

0 Karma

reach2tushar
Explorer

I am using full enterprise instance as a forwarder. Also I verified regex with http://www.regexr.com/. It looks good. The SEDCMD is in props.conf file on indexer.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Since you are using a heavy forwarder, put the props the forwarder

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

reach2tushar
Explorer

It worked after moving on heavy forwarder.
Thanks heaps for your help.

0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...