topic Re: How to remove information from forwarded data? in Getting Data In

How to remove information from forwarded data?

ddarmand — Mon, 19 Oct 2015 09:20:18 GMT

Hello everyone,

This is my topology:

Splunk Forwarder (with local copy of data) -----> Main Splunk

The forwarder is adding sourcetype from regex etc... and it appears in the main Splunk, but I prefer to have raw data. How is it possible to have this without deleting rules on the Splunk forwarder?

Thanks,

Damien

Re: How to remove information from forwarded data?

asimagu — Mon, 19 Oct 2015 09:45:58 GMT

so you are using a HW Forwarder?? why don't you try using a UF?

Re: How to remove information from forwarded data?

ddarmand — Mon, 19 Oct 2015 10:24:35 GMT

because i need to keep local copy of events on the forwarder !

Re: How to remove information from forwarded data?

woodcock — Mon, 19 Oct 2015 13:32:21 GMT

What makes you think that your sourcetype is being added to your raw event? I am unaware of any automatic (i.e. accidental) way that this could happen and to deliberately make it happen, although not difficult, is certainly something that takes some work. I suspect that your raw events are actually OK but you can see for yourself like this:

... | table sourcetype _raw

You will probably see that the field _raw always has an identical/unmodified copy of your raw events.

Re: How to remove information from forwarded data?

asimagu — Mon, 19 Oct 2015 15:30:44 GMT

cool, the more info you provide , the better we will be able to assist 🙂

Re: How to remove information from forwarded data?

ddarmand — Tue, 20 Oct 2015 07:54:10 GMT

It's added because i have some configurations to do this in props.conf transforms.conf ect... but i dont wan't to have these infos on the forwarded data

Re: How to remove information from forwarded data?

woodcock — Tue, 20 Oct 2015 15:07:30 GMT

OK, I am TOTALLY confused. You are deliberately adding stuff to your raw events but you would like to not do so? Back all the way up and FULLY explain your existing configurations (and maybe why they are that way) and the explain your desired end state. As it is, right now, I am utterly confounded.

Re: How to remove information from forwarded data?

ddarmand — Wed, 21 Oct 2015 09:16:08 GMT

Ok, sorry i will try to explain this :

http://zupimages.net/viewer.php?id=15/43/mnyy.png

As you can see, we have two types of users A and B.

I want to see on the main splunk for user B the logs without modifications by the heavy forwarder (as they come by syslog)

These modifications is adding sourcetype for example with props.conf and transforms.conf ect...

But user A need these modifications.

Re: How to remove information from forwarded data?

woodcock — Wed, 21 Oct 2015 16:42:55 GMT

If I understand you correctly, this is your situation:

We have two types of users: A and B both generating the same events that are coming through syslog.  For type-B users, we'd like the logs without modifications (added fields, e.g. sourcetype) by the heavy forwarder (just as they come by syslog).  For type-A users, we'd like to add/keep these modifications.

If this is correct, then you need to build a REGEX that can match type-A users but not type-B users. Once this is done, you need to modify the stanza in transforms.conf that is adding the fields so that it has a REGEX= line. That should do it.