topic Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder? in Getting Data In

How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler — Sat, 06 Jun 2020 23:31:25 GMT

I want to monitor log files and some custom files from /tmp/log_folder on a linux server.

On the Linux box, the desired logs are scripted to /tmp/log_folder/ and this folder will be monitored by the UF.

There is a script to clear out the folder every hour, any file older than 1 day.

So far, I installed a UF on the server.

Besides creating an inputs app (inputs.conf) on the UF and adding the monitoring stanza

 [monitor///tmp/log_folder/*] 
index=special_logs
sourcetype = log_sourcetype
ignoreOlderThan = 1d

Do I need to add anything else?

Thank you

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

ddrillic — Mon, 02 Apr 2018 19:11:16 GMT

The outputs.conf should point to your indexers and the special_logs index should exist.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

skoelpin — Mon, 02 Apr 2018 20:05:15 GMT

Bingo! Once you configure outputs.conf and restart the Splunkd service on the UF, logs will start flowing into Splnuk

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

splunker12er — Tue, 03 Apr 2018 02:33:46 GMT

To monitor log files under a folder execute the command : (or create inputs.conf)

./splunk add monitor /tmp/log_folder/

To forward logs to Splunk Indexer: (outputs.conf)

./splunk add forward-server <splunk-indexer>:<port>

restart splunk services on the forwarder and search for logs.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

MuS — Tue, 03 Apr 2018 02:40:26 GMT

your monitor stanza is missing a : it should be [monitor:///tmp/log_folder/*]
also, don't forget to grant the user running Splunk read and execute permission on /tmp/log_folder/

cheers, MuS

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler — Wed, 04 Apr 2018 15:49:13 GMT

Thank you for noting the error and advising about the permissions.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler — Wed, 04 Apr 2018 15:52:00 GMT

Thank you for reminding me to create the outputs app (outputs.conf), which I am actually hopping thru an HF first. The HF is configure to send to indexers.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler — Wed, 04 Apr 2018 15:53:05 GMT

Thank you for the reply and instructions.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Venkat_16 — Wed, 04 Apr 2018 15:55:28 GMT

you also have to create a file called outputs.conf

[tcpout]
defaultGroup = default group

[tcpout:default group ]
server = indexer_ipaddress:port

also make sure the port 9997 is open in the indexer settings

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

Log_wrangler — Wed, 04 Apr 2018 15:58:27 GMT

I like the simplicity of this way to get the inputs and outputs created.

Re: How to monitor log files from /tmp/folder_name with a Universal Forwarder?

ldongradi_SPL — Thu, 04 Jun 2020 11:57:52 GMT

/tmp/ folder can't be natively monitored by splunk as the splunkd process does not have permissions to access your files in /tmp/

You'd either need to have the files in /tmp generated by splunkd, or give extra permissions to the splunkd process owner to access /tmp files