Getting Data In

How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler
Builder

I want to monitor log files and some custom files from /tmp/log_folder on a linux server.

On the Linux box, the desired logs are scripted to /tmp/log_folder/ and this folder will be monitored by the UF.

There is a script to clear out the folder every hour, any file older than 1 day.

So far, I installed a UF on the server.

Besides creating an inputs app (inputs.conf) on the UF and adding the monitoring stanza

 [monitor///tmp/log_folder/*] 
index=special_logs
sourcetype = log_sourcetype
ignoreOlderThan = 1d

Do I need to add anything else?

Thank you

Labels (2)
0 Karma
1 Solution

splunker12er
Motivator

To monitor log files under a folder execute the command : (or create inputs.conf)

./splunk add monitor /tmp/log_folder/

To forward logs to Splunk Indexer: (outputs.conf)

./splunk add forward-server <splunk-indexer>:<port>

restart splunk services on the forwarder and search for logs.

View solution in original post

ldongradi_splun
Splunk Employee
Splunk Employee

/tmp/ folder can't be natively monitored by splunk as the splunkd process does not have permissions to access your files in /tmp/

You'd either need to have the files in /tmp generated by splunkd, or give extra permissions to the splunkd process owner to access /tmp files

0 Karma

Venkat_16
Contributor

you also have to create a file called outputs.conf

[tcpout]
defaultGroup = default group

[tcpout:default group ]
server = indexer_ipaddress:port

also make sure the port 9997 is open in the indexer settings

0 Karma

MuS
SplunkTrust
SplunkTrust

your monitor stanza is missing a : it should be [monitor:///tmp/log_folder/*]
also, don't forget to grant the user running Splunk read and execute permission on /tmp/log_folder/

cheers, MuS

0 Karma

Log_wrangler
Builder

Thank you for noting the error and advising about the permissions.

0 Karma

splunker12er
Motivator

To monitor log files under a folder execute the command : (or create inputs.conf)

./splunk add monitor /tmp/log_folder/

To forward logs to Splunk Indexer: (outputs.conf)

./splunk add forward-server <splunk-indexer>:<port>

restart splunk services on the forwarder and search for logs.

View solution in original post

Log_wrangler
Builder

I like the simplicity of this way to get the inputs and outputs created.

0 Karma

ddrillic
Ultra Champion

The outputs.conf should point to your indexers and the special_logs index should exist.

skoelpin
SplunkTrust
SplunkTrust

Bingo! Once you configure outputs.conf and restart the Splunkd service on the UF, logs will start flowing into Splnuk

0 Karma

Log_wrangler
Builder

Thank you for the reply and instructions.

0 Karma

Log_wrangler
Builder

Thank you for reminding me to create the outputs app (outputs.conf), which I am actually hopping thru an HF first. The HF is configure to send to indexers.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!