Getting Data In

How to Filter part of data in an event during index time

reach2tushar
Explorer

Hi, I have a type of following event data which is coming from forwarder:

Column1=XYZ+Column2=ABC+ColumnC=GGG....

I want to remove Column2=ABC value from the event before indexing. Can help how to filter this data. The event should be indexed like this:

Column1=XYZ+ColumnC=GGG....
0 Karma

diogofgm
SplunkTrust
SplunkTrust

Try this:
props.conf

[yoursource_type]
SEDCMD-removecolumn=s/Column2=[^\+]*\+//g
------------
Hope I was able to help you. If so, some karma would be appreciated.

reach2tushar
Explorer

Hi, thanks for the reply.
I have windows server environment. I tried this but it didn't work.
will SEDCMD only work on Linux server?

0 Karma

somesoni2
Revered Legend

This should work regardless of OS. Did you put the config in your heavy forwarder/indexers and restarted it? Also, it would be good if you can test the regex against your actual data from www.regex101.com or similar sites before actually using it.

0 Karma

reach2tushar
Explorer

I added these configs on my indexer and restarted. should I move the configs to forwarder?

0 Karma

somesoni2
Revered Legend

What type of forwarder you've, Universal forwarder OR full Enterprise Instance acting as forwarder? Also, Since you posted dummy data in question, the regex is suggested accordingly. Do remember to validate the regex first (if regex is wrong, the SEDCMD will not work anyways).

0 Karma

reach2tushar
Explorer

I am using full enterprise instance as a forwarder. Also I verified regex with http://www.regexr.com/. It looks good. The SEDCMD is in props.conf file on indexer.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Since you are using a heavy forwarder, put the props the forwarder

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

reach2tushar
Explorer

It worked after moving on heavy forwarder.
Thanks heaps for your help.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...