Deployment Architecture

Reducing a light forwarder's footprint using a config file?

dpadams
Communicator

I'm trying out Splunk for the first time and have a question about how to reduce the footprint of a light forwarder. For background, I'm consolidating some Apache logs from eight load-balanced machines onto a single Splunk server. All of the machines are running the current version of Splunk on Windows (a 2008 data server version.) All of the logs are being sent over TCP to the central Splunk server - I don't have any scripted inputs, event log inputs, etc.

I found an interesting entry in the wiki regarding minimizing the forwarder footprint by changing the queue size and disabling some apps.

http://www.splunk.com/wiki/Community:MinimizingForwarderFootprint

The entry mentions a configuration file named that doesn't seem to exist on my install:

etc/apps/SplunkLightForwarder/default/setup.conf

What I'm trying to figure out is how to disable unused features like scripted inputs, file system change monitoring, and windows event logs. Since I've got several identical machines to configured, I'd like to put the settings into a config file, just as I am with the inputs and outputs:

C:\Program Files\Splunk\etc\system\local\inputs.conf C:\Program Files\Splunk\etc\system\local\outputs.conf

Any guidance on what directives to put and where to put them would be most appreciated. I'm new to Splunk but really exciteda bout getting to know it better.

Tags (1)
0 Karma
2 Solutions

MarioM
Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true

View solution in original post

0 Karma

MarioM
Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true

View solution in original post

0 Karma

dpadams
Communicator

Thanks very much for the help!

0 Karma

MarioM
Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true
0 Karma

MarioM
Motivator

Thanks Dwaddle for tidying up my post!

0 Karma

MarioM
Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true
0 Karma

MarioM
Motivator

different .conf on your system/local to reduce footprint.

default-mode.conf:

[pipeline:indexerPipe]

disabled_processors= indexandforward, diskusage, signing,tcp-output-generic-processor, syslog-output-generic-processor, http-output-generic-processor, stream-output-processor  

[pipeline:distributedDeployment]
disabled = true

[pipeline:distributedSearch]
disabled = true

[pipeline:fifo]
disabled = true

[pipeline:merging]
disabled = true

[pipeline:typing]
disabled = true

[pipeline:udp]
disabled = false

[pipeline:tcp]
disabled = false

[pipeline:syslogfifo]
disabled = true

[pipeline:syslogudp]
disabled = true

[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut 


[pipeline:scheduler]
disabled_processors = LiveSplunks 
0 Karma

MarioM
Motivator

as well there is no linefeed between stanza and value just it doesnot display properly in my post

0 Karma

MarioM
Motivator

yes in system/local and yes once in local nothing will be change during update

0 Karma

dpadams
Communicator

Thanks for the quick answer and all of the details. I'm too new to Splunk to follow all of what it means. I'm planning to use a customized outputs.conf and inputs.conf. If I read your answers correctly:
* I'd include the lines you've listed for inputs.conf in my system/local inputs.conf.
* I'd add the indexes.conf and default-mode conf files you've listed.

If I've got that right, where should those two files indexes & default-mode) be placed? system/local? My understanding is that files in that location are not changed by Splunk during an update.

Thanks very much for any additional help!

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...