topic forwarding too slow in Getting Data In

forwarding too slow

crazyeva — Wed, 25 Dec 2013 04:28:45 GMT

i let universalforwarder distribute raw data, when doing unzip work;
indexer doing sedcmd-filter, transfrom-filter work
But indexing process very slow 500events/persecond
cpu, diskio, network all at low utility
i dont know what is wrong
Can i just:
# cp /opt/splunkforwarder /opt/splunkforwarder01
# /opt/splunkforwarder01/bin/./splunk start
#
# cp /opt/splunkforwarder /opt/splunkforwarder02
# /opt/splunkforwarder02/bin/./splunk start
.....
?

Re: forwarding too slow

martin_mueller — Wed, 25 Dec 2013 22:24:58 GMT

I don't see any benefit in running several copies of the same forwarder on one machine.

As for forwarding speed, make sure you aren't hitting the configurable speed limiter, I believe the default is 256KB/s - if your events are 500byte on average then 500eps is about 256KB/s.

Re: forwarding too slow

crazyeva — Thu, 26 Dec 2013 01:58:41 GMT

You are quite right! maxKBps was set to 256!
Its the first time I use universalforwarder.
Thank you very much!