Do you have any recommendations for Universal forwarder settings that would ease the resource usage for Universal Forwarders loaded on AWS micro servers?

[jpt751]

One of our user applications utilizes over 50% Micro Servers in AWS. The micros meet the minimum requirements for Splunk, but experienced high CPU usage once the Universal forwarders instances were added to them. These micros are being used to host static web pages. Do you have any recommendations for Universal forwarder settings that would ease the resource usage? Or do you have any suggestions for an alternate way to extract the logs from the micros?

[lguinn2]

Generally, the CPU usage of the the Universal Forwarder (UF) is pretty directly tied to the number of files being monitored. Quite often, the UF is pointed at a directory of log files - and a lot of the files are stale. You can often boost UF performance by writing a simple script (or using the logrotate command in Linux) to move stale files to an archive directory - or delete them.

One of the other issues with the AWS micro issues may be the network performance. I quit using micro instances as much as possible due to the low network performance. This also can have an effect on Splunk and the networking infrastructure in general. This was a problem in my particular case even though I did not have a high data volume. If you are not monitoring a lot of files, try setting up an instance with better network performance and see if the problem goes away.

I don't know the exact network performance specs for the various AWS instances, but I am pretty sure that micro instances don't provide the equivalent of a 1 GB NIC.

[nkwong_splunk]

Are you using t1.micro instances? If so, I'd recommended trying the newer t2.micro instances since it has better baseline performance, burstable performance, and they are cheaper.