Getting Data In

How to assess improvement in network utilization after turning on compression on a universal forwarder?

JeremyHagan
Communicator

Hi,

I'm wanting to assess the improvement in network utilization after turning on compression. Is there any search of the internal index or metrics that are collected that can allow me to assess the improvement?

Also, when I configure the outputs.conf on my Universal Forwarder to the following:

[tcpout]
defaultGroup = default-autolb-group
compressed = true

and restarted, I expected that events would stop being indexed until I configured the inputs.conf on the indexer like this:

[splunktcp://9997]
connection_host = ip
compressed = true

But data continued to flow. Now I am worried that compression isn't actually working. How do I tell?

ryandg
Communicator

What does the source of the log say? is it a gz file? I am wondering if the compressed=true isn't necessary, I have noticed that with non-Splunk compressed (but they are GZ files) files will automatically get uncompressed at index.

EDIT: After searching it looks like if you have SSL enabled, compression is turned on by default - meaning that the compressed flag does nothing.

EDIT2: Have you checked to make sure that the outputs.conf is the correct one? Have you checked using btool?

0 Karma

JeremyHagan
Communicator

I've since worked out what btool was and using btool inputs list and btool output list I see the compressed = true in both

0 Karma

ryandg
Communicator

Interesting, the only true way you could check this is if you have a network monitoring application installed that would allow you to see outbound and inbound from the splunk box.

The other option which may or may not work is you should see CPU increase when compression is turned on versus off -- depending on how many logs you are getting that it needs to compress the average CPU should increase by 1-2%. If you have SOS installed and sending to the indexers or the DMC enabled for this box you could actually just monitor between the two settings for a few minutes and check to see if the CPU jumps up a bit above where it is when compressed is commented out.

0 Karma

JeremyHagan
Communicator

The outputs.conf file is the one in /etc/system/local

We don't have SSL enabled.

The only source enabled on the source host is the Windows Event Log.

I've not heard of btool. What is that? I'll see if I can find out.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...