Is it possible to prioritize what data is forwarded from a heavy forwarder? (ex: security data first, then non-security data)

[dkeck]

Hi everybody,

I'm new in Splunk, so be gentle, please.

So that's the scenario:

I have a Splunk Heavy forwarder, and I want to know if it is possible to prioritize the data which is forwarded to the indexer(s)?

For example: I have security relevant log data and I want this data to be forwarded first, every time. So that non-security relevant data is held back until the security relevant data is indexed.

Is that possible and how?

If possible, looking for solutions which are built-in to out-of-box Splunk, add-ons etc. I can't use another software for it since the system Splunk is running on is already pretty limited.

Thank you

David

[ryandg · Accepted solution]

The only answer I have is one that you won't really benefit from. We had Splunk Pro Services visit us and one of the questions that came up was this exact question. Splunk does not have any prioritization ability according to them, hopefully someone will correct me if this was wrong but that is what we were told.

[dkeck]

Nobody has an answer? too bad.

[Richfez]

What problems are you experiencing that you feel needs to be solved with prioritization of certain logs/data over other logs/data? Are you having problems out of your indexer, since it's already at the edge of overloaded?

But a thought, not perfect but at least something - if you ran two copies of Splunk as forwarders, you could prioritize the processes themselves, cranking one way down to "idle" time processing and the other leaving at normal. That could make a minor change in how each would behave.

[jeffland]

Hi, allow me to chime in. The problem is that our bandwidth is very limited (also entirely unavailable for extended time periods), thus we'd like to be able to send the priority data first as soon as we have a (stable) connection. Only after that has been sent should the less important data be forwarded.
We'll be using indexer acknowledgement, so this could also be applied at the forwarding buffer level: we'd like to keep more of the important data than of the less important data queued on the forwarder, and if needed we'd also replace the queued unimportant data with the important data (we are really limited in this environment, otherwise we'd simply keep everything on the HF).

[Richfez]

Thanks, that explanation of why you need it will probably help folks trying to answer this!