Deployment Architecture

Reducing a light forwarder's footprint using a config file?

Communicator

I'm trying out Splunk for the first time and have a question about how to reduce the footprint of a light forwarder. For background, I'm consolidating some Apache logs from eight load-balanced machines onto a single Splunk server. All of the machines are running the current version of Splunk on Windows (a 2008 data server version.) All of the logs are being sent over TCP to the central Splunk server - I don't have any scripted inputs, event log inputs, etc.

I found an interesting entry in the wiki regarding minimizing the forwarder footprint by changing the queue size and disabling some apps.

http://www.splunk.com/wiki/Community:MinimizingForwarderFootprint

The entry mentions a configuration file named that doesn't seem to exist on my install:

etc/apps/SplunkLightForwarder/default/setup.conf

What I'm trying to figure out is how to disable unused features like scripted inputs, file system change monitoring, and windows event logs. Since I've got several identical machines to configured, I'd like to put the settings into a config file, just as I am with the inputs and outputs:

C:\Program Files\Splunk\etc\system\local\inputs.conf C:\Program Files\Splunk\etc\system\local\outputs.conf

Any guidance on what directives to put and where to put them would be most appreciated. I'm new to Splunk but really exciteda bout getting to know it better.

Tags (1)
0 Karma
2 Solutions

Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true

View solution in original post

0 Karma

Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true

View solution in original post

0 Karma

Communicator

Thanks very much for the help!

0 Karma

Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true

View solution in original post

0 Karma

Motivator

Thanks Dwaddle for tidying up my post!

0 Karma

Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true

View solution in original post

0 Karma

Motivator

different .conf on your system/local to reduce footprint.

default-mode.conf:

[pipeline:indexerPipe]

disabled_processors= indexandforward, diskusage, signing,tcp-output-generic-processor, syslog-output-generic-processor, http-output-generic-processor, stream-output-processor  

[pipeline:distributedDeployment]
disabled = true

[pipeline:distributedSearch]
disabled = true

[pipeline:fifo]
disabled = true

[pipeline:merging]
disabled = true

[pipeline:typing]
disabled = true

[pipeline:udp]
disabled = false

[pipeline:tcp]
disabled = false

[pipeline:syslogfifo]
disabled = true

[pipeline:syslogudp]
disabled = true

[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut 


[pipeline:scheduler]
disabled_processors = LiveSplunks 
0 Karma

Motivator

as well there is no linefeed between stanza and value just it doesnot display properly in my post

0 Karma

Motivator

yes in system/local and yes once in local nothing will be change during update

0 Karma

Communicator

Thanks for the quick answer and all of the details. I'm too new to Splunk to follow all of what it means. I'm planning to use a customized outputs.conf and inputs.conf. If I read your answers correctly:
* I'd include the lines you've listed for inputs.conf in my system/local inputs.conf.
* I'd add the indexes.conf and default-mode conf files you've listed.

If I've got that right, where should those two files indexes & default-mode) be placed? system/local? My understanding is that files in that location are not changed by Splunk during an update.

Thanks very much for any additional help!

0 Karma