topic Re: What is the best way to filter events at Heavy Forwarder level? in Getting Data In

What is the best way to filter events at Heavy Forwarder level?

aoliullah — Mon, 24 Apr 2017 20:41:22 GMT

Hi.

I am trying to send logs from a bunch of Universal Forwarders (UF) to a Heavy Forwarder which will then forward it to a SOC (managed service - we have a syslog receiver onsite).

Currently, all the logs are being indexed into Splunk but I am planning to edit the outputs stanza on the UFs by adding another group with the Heavy Forwarder's IP address, so that it creates a data clone and then I can filter out the required data at the HF before sending it SOC.

I am trying to figure out the best method of filtering this data. Basically, these UFs are monitoring lots of application data in addition to the local event logs and other security logs. I am only interested in the local event logs (both Windows and Unix) and security logs and want to get rid of all other logs (nullQueue).

What would be the best way to achieve this? Should I filter using the source (i.e. Whitelisting a number of sources)? So that only the whitelisted sources are forwarded by the HF to the SOC and all the rest from the data clone goes to nullqueue.

Would highly appreciate if someone could show me a config example.

Thanks in advance?

Re: What is the best way to filter events at Heavy Forwarder level?

adonio — Tue, 25 Apr 2017 01:36:21 GMT

hello aoliullah,
are the HF a requirements? you can filter at the UF level or Indexer level. will recommend against HF unless you really have to have it.

Re: What is the best way to filter events at Heavy Forwarder level?

aoliullah — Tue, 25 Apr 2017 04:37:28 GMT

Unfortunately I'm having to route through it. Have to follow the implemented design. Any advice would be appreciated.

Re: What is the best way to filter events at Heavy Forwarder level?

adonio — Tue, 25 Apr 2017 11:55:10 GMT

If you must use the Heavy Forwarder,
use props and transforms as explained in docs: http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad
also many more answers in this portal, look for filter, route, props, transforms, etc.
hope it helps

Re: What is the best way to filter events at Heavy Forwarder level?

NeharikaVats — Tue, 21 Nov 2023 12:29:38 GMT

I want to filter the palo logs at the forwarder level by looking at the packet before indexing( licensing) based certain condition like... zone, firewall name(enterprise) etc

The logs comes to both our UF & HF, what is the best way to achieve it.

Was looking into a few doc suggesting to apply ingest eval, is that feasible?

Can anyone please help me with this.

Re: What is the best way to filter events at Heavy Forwarder level?

PickleRick — Tue, 21 Nov 2023 13:35:26 GMT

You're much more likely to get a relevant answer if you post a new question instead of digging up an old thread (especially that old).