topic Filter data in Getting Data In

Filter data

aalaa — Wed, 10 Jul 2019 08:26:13 GMT

Hello ,
I need to filter data in a heavy forwarder ,
by discarding some of event : i have the field "id" in my data this field contains many type , i need to discard the id type id="1200006"

So , i try with this :

Propos.conf :
[source::tcp:516]
TRANSFORMS-null= setnull

transforms.conf :
[setnull]
REGEX =[.1200006.]
DEST_KEY = queue
FORMAT = nullQueue

but it does not give a result !
Any help please , Thank you

Re: Filter data

FrankVl — Wed, 10 Jul 2019 09:08:01 GMT

That regex is incorrect. Square brackets are for defining character sets. In this case your regex will match any event containing literal dots, 1, 2, 0 or 6. So it would likely send pretty much all your events to the nullQueue. Just use REGEX = id="1200006".

You say "it does not give a result !". Do you mean you don't see any events anymore (explained by incorrect regex) or do you mean the config seems to have no effect? In that latter case: have you restarted the HF after adding that config? Have you checked using btool that that config is correctly interpreted by Splunk? Perhaps conflict with other transforms (setnull is not a very unique name and it must be a unique name).

Re: Filter data

aalaa — Wed, 10 Jul 2019 09:36:17 GMT

i try with this :

Transofrms.conf :
[setnull]
REGEX = id ="1200006"
DEST_KEY = queue
FORMAT = nullQueue

Propos.conf :
[source::tcp://516]
TRANSFORMS-null= setnull

same result , the event with the id=1200006 rest exist

Re: Filter data

FrankVl — Wed, 10 Jul 2019 09:39:45 GMT

Then have a look at the second part of my answer. And I think you can keep the [source::tcp:516] as you had it initially.