topic Re: Filter specific data to indexer in Getting Data In

Filter specific data to indexer

SplunkCSIT — Wed, 16 Apr 2014 03:28:54 GMT

Hi,
i do not want to forward the body and the content field to indexer, how to go abt doing it? thks

<xml>
    <Field1>123</Field1>
    <Field2>456</Field2>
    <Body>Not to be forward to indexer</Body>
    <Field3>333</Field3>
    <content>not to be forward</content>
</xml>

Re: Filter specific data to indexer

linu1988 — Wed, 16 Apr 2014 05:39:35 GMT

first of all you can't filter this not to be forwarded from forwarder. It need to be done on the indexer. Not sure as it is a single event how to apply sedcmd on this

Re: Filter specific data to indexer

MuS — Wed, 16 Apr 2014 05:43:24 GMT

Hi SplunkCSIT,

take a look at the docs about Filter event data and send to queues. If the wanted and unwanted fields are all in one event, you may need to use this example and instead of replacing the data you blank them out.

In your case you can either set the regex to something like this <Field\d+>(.+)<\/Field\d+> to match and keep only the Field events or set the regex to match (<Body>|<content>)(.+)<\/ and have that removed.

The regex's are untested because I don't have the real world event, so maybe they need some tuning.

Update: and like @linu1988 said, this must be done on a parsing Splunk instance like a Indexer and/or a heavy forarwder.

hope this helps ...

cheers, MuS

Re: Filter specific data to indexer

SplunkCSIT — Thu, 17 Apr 2014 00:17:06 GMT

the props.conf and transforms.conf to be configured at forwarder or indexer?

Re: Filter specific data to indexer

SplunkCSIT — Thu, 17 Apr 2014 02:08:42 GMT

at props.conf,
[test]
TRANSFORMS-null = content

at transforms.conf,
[content]
REGEX = (.+)<\/content>
DEST_KEY = queue
FORMAT = nullQueue

I configured the props.conf and transforms.conf for both forwarder and indexer but it does not work. Any problem with my config files?

Re: Filter specific data to indexer

SplunkCSIT — Thu, 17 Apr 2014 16:23:44 GMT

someone had provided me the ans.it worked

Re: Filter specific data to indexer

tararso — Thu, 17 Apr 2014 17:03:16 GMT

props.conf and transforms.conf needs to configured on indexer, not at Universal forwarder. If you have heavy forwarder, you can use props.conf and transforms.conf.