Does Splunk have a precedence for what logs to forwarder first?

[EatMoreChicken]

If I had logs for the `_internal` index and logs for a `linux_os` index on a Heavy Forwarder, does the HF prioritize the `linux_os` index data prior to the `_internal` data on the host? Is there any precedence for data Splunk is monitoring?  Does Indexers have a precedence for what kind of data to index first?

[PickleRick]

There is a question of what do you mean by priority because there are at least two different situations whent it might make a difference.

One is a question which data is getting enqueued and/or transmitted first when the forwarder cannot send the data with full speed (receiving party stalling or thruput limits). And here indeed _internal logs might be prioritized. But to be fully honest, I don't see a good reason for that. After all those are only normal monitor inputs watching files in $SPLUNK_HOME/var/log/splunkd

But another situation is if you (re)start the forwarder and it has to restart all inputs which means it has to re-examine all monitored files, re-read them and so in. In this case - from my experience - there is definitely no _internal priority. I had many situations with UF's monitoring huge sets of files that I would have to wait up to several hours for my _internal events after restart.

[gcusello]

Hi @EatMoreChicken,

for my knowledge, it isn't possible define a priority between logs, with the only exception on the Splunk internal logs.

So, in your case linux_os logs are surely sent before _internal, but it isn't possible to prioritize linux_os respect wineventlog.

Ciao.

Giuseppe

[EatMoreChicken]

Gotcha, that's exactly what I was seeing. `_internal` logs were consistently coming in after other logs, so just wanted to make sure. Would you happen to know of any Splunk docs documenting this? Just wondering.

[gcusello]

Hi @EatMoreChicken,

why you're wondering? it's a logical behavior.

I searched some doc about this but Ididn't find it soon, sorry.

Ciao.

Giuseppe

[EatMoreChicken]

Just wanted to have some concrete documentation around it in the event someone else asks me.