Do shortened events from universal forwarder count towards license?

[Som]

Hi,

Our event size is set to the default 10,000 bytes. We are using the universal forwarder to get log events to our indexing machine.

However, we have some log lines that output 1-200KB of data. It's okay for this data to be shortened to 10,000 bytes, but I'm curious how many bytes are being counted towards our daily license in this setup. Is it the 10,000 bytes? Or is it the full 1-200KB of data?

I guess another thing I'm not clear on is -- does the shortening happen on the forwarder side or the indexer side?

Thanks for any insight you can provide!

Som

[gcusello · Accepted solution]

Hi @Som,

the Splunk license consuption is counted on the daily really indexed logs, in other words, if you filter events before indexing, in the license consuption there are only the indexed logs, not the original dimesnion.

So in your case, if you have a filter to 10K on events, you index only until 10K for events and this is your consuption, obviously the content of the other part of logs is loosed and you cannot use it.

About filtering, there are filters applied on Forwarders and some others applied on Indexers, in your case the filter is applied on Forwarders.

Ciao.

Giuseppe

[Som]

Ah, okay, thank you!

To confirm -- the default 10K limit for event sizes is something that the basic install of the universal forwarder would apply. I ask because I'm assuming this means the universal forwarder installs with some kind of filter by default, as I don't believe I've set up any kind of filtering on it.

Thanks again for your help!

Som

[gcusello]

Hi @Som,

there isn't any default limitation in Forwarders installation, you can truncate events in your configurations.

If this answer solves your need, please accept it for the other people of Community.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

[Som]

I think that's the part I'm confused about then. Because I haven't made any changes to truncate events from the forwarder, but I'm still getting truncated events. According to several other results (here's one: https://community.splunk.com/t5/Knowledge-Management/What-is-the-maximum-length-of-a-tag-and-an-even...), this is default behavior and Splunk's default event size is 10K.

But if you're saying that the forwarder doesn't truncate by default, then that is telling me that the entire log is sent to the indexer and then truncated on the indexer side. Which still leaves my question open -- is the full size being counted towards the license? Or the post-truncated 10K?

Som

[gcusello]

Hi @Som,

the limit of 10K isn't in the Forwarder but in the Indexers, as you can see in https://docs.splunk.com/Documentation/Splunk/8.2.0/Admin/Propsconf (TRUNCATE).

Ciao.

Giuseppe

[Som]

Ah, ok. Makes sense.

So to confirm, if the indexer is truncating, does the truncated amount go towards the license or the amount sent from the forwarder?

Thanks again! (Will accept your response to this question as I think it’ll be most applicable.)

Som

[gcusello]

Hi @Som,

as I said, in the license consuption there's only indexed logs, so if a log is filtered (on Forwarder or on Indexers) before indexing,  the discarded logs aren't countered in the license consuption, but remeber that these logs are loosed for searches, so think with attention to which logs can be truncated and which not.

Tell me if i can help you againg. and see next time.

Ciao and happy splunking.

Giuseppe