Deployment Architecture

Reducing a light forwarder's footprint using a config file?

dpadams
Communicator

I'm trying out Splunk for the first time and have a question about how to reduce the footprint of a light forwarder. For background, I'm consolidating some Apache logs from eight load-balanced machines onto a single Splunk server. All of the machines are running the current version of Splunk on Windows (a 2008 data server version.) All of the logs are being sent over TCP to the central Splunk server - I don't have any scripted inputs, event log inputs, etc.

I found an interesting entry in the wiki regarding minimizing the forwarder footprint by changing the queue size and disabling some apps.

http://www.splunk.com/wiki/Community:MinimizingForwarderFootprint

The entry mentions a configuration file named that doesn't seem to exist on my install:

etc/apps/SplunkLightForwarder/default/setup.conf

What I'm trying to figure out is how to disable unused features like scripted inputs, file system change monitoring, and windows event logs. Since I've got several identical machines to configured, I'd like to put the settings into a config file, just as I am with the inputs and outputs:

C:\Program Files\Splunk\etc\system\local\inputs.conf C:\Program Files\Splunk\etc\system\local\outputs.conf

Any guidance on what directives to put and where to put them would be most appreciated. I'm new to Splunk but really exciteda bout getting to know it better.

Tags (1)
0 Karma
2 Solutions

MarioM
Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true

View solution in original post

0 Karma

MarioM
Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true

View solution in original post

0 Karma

dpadams
Communicator

Thanks very much for the help!

0 Karma

MarioM
Motivator

inputs.conf:

[monitor://$SPLUNK_HOME\var\log\splunk\web_access.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\web_service.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\searchhistory.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunklogger.log]
disabled = true

[fschange:$SPLUNK_HOME\etc]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\splunkd.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
disabled = true

[monitor://$SPLUNK_HOME\var\log\splunk\license_audit.log]
disabled = true
0 Karma

MarioM
Motivator

Thanks Dwaddle for tidying up my post!

0 Karma

MarioM
Motivator

indexes.conf:

maxConcurrentOptimizes = 1
defaultDatabase = _thefishbucket
blockSignatureDatabase = _thefishbucket

[main]
disabled = true

[history]
disabled = true

[summary]
disabled = true

[_internal]
disabled = true

[_audit]
disabled = true

[_blocksignature]
disabled = true
0 Karma

MarioM
Motivator

different .conf on your system/local to reduce footprint.

default-mode.conf:

[pipeline:indexerPipe]

disabled_processors= indexandforward, diskusage, signing,tcp-output-generic-processor, syslog-output-generic-processor, http-output-generic-processor, stream-output-processor  

[pipeline:distributedDeployment]
disabled = true

[pipeline:distributedSearch]
disabled = true

[pipeline:fifo]
disabled = true

[pipeline:merging]
disabled = true

[pipeline:typing]
disabled = true

[pipeline:udp]
disabled = false

[pipeline:tcp]
disabled = false

[pipeline:syslogfifo]
disabled = true

[pipeline:syslogudp]
disabled = true

[pipeline:parsing]
disabled_processors=utf8, linebreaker, header, sendOut 


[pipeline:scheduler]
disabled_processors = LiveSplunks 
0 Karma

MarioM
Motivator

as well there is no linefeed between stanza and value just it doesnot display properly in my post

0 Karma

MarioM
Motivator

yes in system/local and yes once in local nothing will be change during update

0 Karma

dpadams
Communicator

Thanks for the quick answer and all of the details. I'm too new to Splunk to follow all of what it means. I'm planning to use a customized outputs.conf and inputs.conf. If I read your answers correctly:
* I'd include the lines you've listed for inputs.conf in my system/local inputs.conf.
* I'd add the indexes.conf and default-mode conf files you've listed.

If I've got that right, where should those two files indexes & default-mode) be placed? system/local? My understanding is that files in that location are not changed by Splunk during an update.

Thanks very much for any additional help!

0 Karma
Get Updates on the Splunk Community!

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...

Thank You for Celebrating CX Day with Splunk!

Yesterday the entire team at Splunk + Cisco joined the global celebration of CX Day - celebrating our ...