I'm trying out Splunk for the first time and have a question about how to reduce the footprint of a light forwarder. For background, I'm consolidating some Apache logs from eight load-balanced machines onto a single Splunk server. All of the machines are running the current version of Splunk on Windows (a 2008 data server version.) All of the logs are being sent over TCP to the central Splunk server - I don't have any scripted inputs, event log inputs, etc.
I found an interesting entry in the wiki regarding minimizing the forwarder footprint by changing the queue size and disabling some apps.
http://www.splunk.com/wiki/Community:MinimizingForwarderFootprint
The entry mentions a configuration file named that doesn't seem to exist on my install:
etc/apps/SplunkLightForwarder/default/setup.conf
What I'm trying to figure out is how to disable unused features like scripted inputs, file system change monitoring, and windows event logs. Since I've got several identical machines to configured, I'd like to put the settings into a config file, just as I am with the inputs and outputs:
C:\Program Files\Splunk\etc\system\local\inputs.conf C:\Program Files\Splunk\etc\system\local\outputs.conf
Any guidance on what directives to put and where to put them would be most appreciated. I'm new to Splunk but really exciteda bout getting to know it better.
indexes.conf:
maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket
[main]
disabled = true
[history]
disabled = true
[summary]
disabled = true
[_internal]
disabled = true
[_audit]
disabled = true
[_blocksignature]
disabled = true
inputs.conf:
[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true
[fschange:$SPLUNK_HOME\etc]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true
Thanks very much for the help!
inputs.conf:
[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true
[fschange:$SPLUNK_HOME\etc]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true
[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true
Thanks Dwaddle for tidying up my post!
indexes.conf:
maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket
[main]
disabled = true
[history]
disabled = true
[summary]
disabled = true
[_internal]
disabled = true
[_audit]
disabled = true
[_blocksignature]
disabled = true
different .conf on your system/local to reduce footprint.
default-mode.conf:
[pipeline:indexerPipe]
disabled_processors= indexandforward, diskusage, signing,tcp-output-generic-processor, syslog-output-generic-processor, http-output-generic-processor, stream-output-processor
[pipeline:distributedDeployment]
disabled = true
[pipeline:distributedSearch]
disabled = true
[pipeline:fifo]
disabled = true
[pipeline:merging]
disabled = true
[pipeline:typing]
disabled = true
[pipeline:udp]
disabled = false
[pipeline:tcp]
disabled = false
[pipeline:syslogfifo]
disabled = true
[pipeline:syslogudp]
disabled = true
[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut
[pipeline:scheduler]
disabled_processors = LiveSplunks
as well there is no linefeed between stanza and value just it doesnot display properly in my post
yes in system/local and yes once in local nothing will be change during update
Thanks for the quick answer and all of the details. I'm too new to Splunk to follow all of what it means. I'm planning to use a customized outputs.conf and inputs.conf. If I read your answers correctly:
* I'd include the lines you've listed for inputs.conf in my system/local inputs.conf.
* I'd add the indexes.conf and default-mode conf files you've listed.
If I've got that right, where should those two files indexes & default-mode) be placed? system/local? My understanding is that files in that location are not changed by Splunk during an update.
Thanks very much for any additional help!