Getting Data In

How to Filter part of data in an event during index time

reach2tushar
Explorer

Hi, I have a type of following event data which is coming from forwarder:

Column1=XYZ+Column2=ABC+ColumnC=GGG....

I want to remove Column2=ABC value from the event before indexing. Can help how to filter this data. The event should be indexed like this:

Column1=XYZ+ColumnC=GGG....
0 Karma

diogofgm
SplunkTrust
SplunkTrust

Try this:
props.conf

[yoursource_type]
SEDCMD-removecolumn=s/Column2=[^\+]*\+//g
------------
Hope I was able to help you. If so, some karma would be appreciated.

reach2tushar
Explorer

Hi, thanks for the reply.
I have windows server environment. I tried this but it didn't work.
will SEDCMD only work on Linux server?

0 Karma

somesoni2
Revered Legend

This should work regardless of OS. Did you put the config in your heavy forwarder/indexers and restarted it? Also, it would be good if you can test the regex against your actual data from www.regex101.com or similar sites before actually using it.

0 Karma

reach2tushar
Explorer

I added these configs on my indexer and restarted. should I move the configs to forwarder?

0 Karma

somesoni2
Revered Legend

What type of forwarder you've, Universal forwarder OR full Enterprise Instance acting as forwarder? Also, Since you posted dummy data in question, the regex is suggested accordingly. Do remember to validate the regex first (if regex is wrong, the SEDCMD will not work anyways).

0 Karma

reach2tushar
Explorer

I am using full enterprise instance as a forwarder. Also I verified regex with http://www.regexr.com/. It looks good. The SEDCMD is in props.conf file on indexer.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Since you are using a heavy forwarder, put the props the forwarder

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

reach2tushar
Explorer

It worked after moving on heavy forwarder.
Thanks heaps for your help.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...