If I remove the only input stanza I have on a forwarder and restart Splunk the memory usage is 2GB. How can a forwarder use that much memory if I don't have any input stanzas?
On this page it's described how Splunk use CRC checksums to keep track of which files it has seen before.
How much space does this data use on disk and in memory if you have let Splunk monitor a directory that over the years have contained more than 1 000 000 files?
This question is related to this question:
I had to modify your command since I'm on Windows: %SPLUNKHOME%\bin\splunk.exe cmd btprobe -d %SPLUNKHOME%\var\lib\splunk\fishbucket\splunkprivatedb\ -k ALL | find /c "key". The result is 66827.
Hi andyw. A few questions for you :
- How do you measure the memory consumption of the forwarder? Which metric (Vsize? RSS?) shows the 2Gb figure?
- How large is the directory $SPLUNKHOME/var/lib/splunk/fishbucket on the forwarder?
- How many records exist in your fishbucket? To find out, run :
$SPLUNKHOME/bin/splunk cmd btprobe -d $SPLUNKHOME/var/lib/splunk/fishbucket/splunkprivate_db/ -k ALL | wc -l