Getting Data In

Do you have any recommendations for Universal forwarder settings that would ease the resource usage for Universal Forwarders loaded on AWS micro servers?

jpt751
New Member

One of our user applications utilizes over 50% Micro Servers in AWS. The micros meet the minimum requirements for Splunk, but experienced high CPU usage once the Universal forwarders instances were added to them. These micros are being used to host static web pages. Do you have any recommendations for Universal forwarder settings that would ease the resource usage? Or do you have any suggestions for an alternate way to extract the logs from the micros?

0 Karma

lguinn2
Legend

Generally, the CPU usage of the the Universal Forwarder (UF) is pretty directly tied to the number of files being monitored. Quite often, the UF is pointed at a directory of log files - and a lot of the files are stale. You can often boost UF performance by writing a simple script (or using the logrotate command in Linux) to move stale files to an archive directory - or delete them.

One of the other issues with the AWS micro issues may be the network performance. I quit using micro instances as much as possible due to the low network performance. This also can have an effect on Splunk and the networking infrastructure in general. This was a problem in my particular case even though I did not have a high data volume. If you are not monitoring a lot of files, try setting up an instance with better network performance and see if the problem goes away.

I don't know the exact network performance specs for the various AWS instances, but I am pretty sure that micro instances don't provide the equivalent of a 1 GB NIC.

0 Karma

nkwong_splunk
Splunk Employee
Splunk Employee

Are you using t1.micro instances? If so, I'd recommended trying the newer t2.micro instances since it has better baseline performance, burstable performance, and they are cheaper.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...