Getting Data In

Universal Forwarder not able to read all logs

j_thomas
Explorer

Here is my input.conf:

[monitor:///var/log]
crcSalt =
disabled = false
index = main

From this it should recursively search all logs under /var/log/...

The issue I am seeing is that splunk user is not able to read apache2, audit, and some other logs. Permissions on all those logs that are 0640. Also to note, I have added the splunk user to groups: root, adm and syslog while trying to troubleshoot.

The forwarder should be able to read these files as its apart of the groups, but at this point I am at a loss.

Any help would be greatly appreciated!

UPDATE 11/3/14:

I know this is a little delayed, but the files are still not being read.

The addition of the "..." did help with recursive (and is a partial answer), but the files are still not being ingested/sent. I have performed some testing and maybe you guys can help me see something I don't see.

Current file permissions are 640 and owned by "root:adm". Splunk Forwarder is still apart root, adm and syslog groups. With these file permissions the logs are not being sent, but If i switch to the splunk user I can read the logs without issue. This tells me that the permissions on the logs should be fine and the UF should be able to sent them, correct? But, if i change the ownership to "root:root" the logs start being sent.

Any thoughts on this?

Tags (1)
1 Solution

frmaasdam
Path Finder

I had more or less the same issues.
Logfiles owned by root:adm
Splunk running under user splunk
User splunk member of adm group
Login and su - splunk make it possible to cat the logfiles
BUT
Splunkforwarder running under user splunk was not able to read and forward the logfiles
2 options:
1. Start the Splunkforwarder using su -l splunk -c
This is what I have done
2. Setfacl on the logfiles to make USER splunk able to execute and read the files
I have done this in an other situation

,

View solution in original post

j_thomas
Explorer

That did not resolve the issue either

0 Karma

frmaasdam
Path Finder

There is a bug with GID in Splunk when you autostart splunk. I use su -l splunk -c in the init file.

peter_krammer
Communicator

I think you need the wildcard for recursive monitoring in your config:

[monitor:///var/log/...]
disabled = false
index = main

Please refer to http://docs.splunk.com/Documentation/Splunk/6.1.4/admin/Inputsconf section "Note concerning wildcards and monitor"

j_thomas
Explorer

Thanks Peter!!! This seems to have done it.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...