Deployment Architecture

Is there any way we can discard unwanted log events at universal forwarder level

usha_nittala
New Member

Hi All,

We can do selective forwarding at heavy forwarder or indexer level but i want to know with latest version , is there any possibility of discarding unwanted events in logs ( based on regex pattern) at universal forwarder level. We dont have heavy forwarders so putting lot of conditions in transforms will increase the parsing load on indexer. So want to know if we can configure anything on universal forwarder itself.

Thanks in advance.

0 Karma
1 Solution

FrankVl
Ultra Champion

The basic answer is no, this is specifically one of those use cases you would use Heavy Forwarders for.

Some input methods (e.g. winevtlog) do offer whitelist/blacklist settings though, so you might want to take a look at what type of inputs you have and then check the inputs.conf spec to see if that offers any options.

Also, if you are receiving syslog data, you could use a syslog daemon to do the pre-processing and drop some of the data.

View solution in original post

FrankVl
Ultra Champion

The basic answer is no, this is specifically one of those use cases you would use Heavy Forwarders for.

Some input methods (e.g. winevtlog) do offer whitelist/blacklist settings though, so you might want to take a look at what type of inputs you have and then check the inputs.conf spec to see if that offers any options.

Also, if you are receiving syslog data, you could use a syslog daemon to do the pre-processing and drop some of the data.

masonmorales
Influencer

Upvoted, as this is the correct answer. A universal forward is not capable of performing event-level parsing/filtering operations (aside from Windows Event ID). For a good write-up on UF vs HF, check out: https://www.splunk.com/blog/2016/12/12/universal-or-heavy-that-is-the-question.html

0 Karma

sloshburch
Splunk Employee
Splunk Employee

Also keep in mind that if you end up using a HW you will be causing the data to get more condensed than if it were done at the indexer layer.

Are you actually seeing load issues on the indexers? Before creating more complexity in the environment, we should make sure this is the right feature to play with.

0 Karma

usha_nittala
New Member

Thanks @FrankVl , I will check whitelist/blacklist settings in inputs.conf spec.

0 Karma

deepashri_123
Motivator

Hey@usha_nittala,

Yes you can filter events before indexing. Refer the link below:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Forwarding/Routeandfilterdatad

Let me know if this helps!!

0 Karma

FrankVl
Ultra Champion

@usha_nittala is looking for a solution on UF, that document is mostly about things you can do on HF.

0 Karma

usha_nittala
New Member

Thanks @deepashri_123 and @FrankVl
@FrankVl is right. We don't have enough capacity to add heavy forwarders and we have limited number of indexer servers.We did not want to increase load on indexers and hence wanted to find out if any kind of parsing is done at Universal/light forwarder level.

0 Karma