Getting Data In

Splunk file and directory monitoring- Am I configuring correctly?


Hello having some confusing problems with Splunk permissions that I am trying to understand. Little background we upgrade our index/deployment server from Debian to ubuntu.  

here is the problem I am seeing after this upgrade.


I was monitoring a file in var/log/test-combo.log  and everything worked before hand on debian 11. Now I am not getting any of the data from this file ingested into my index but I can see fresh logs.

The file is owned by syslog and the group is adm.

My splunk user:
uid=1001(splunk) gid=1001(splunk) groups=1001(splunk),4(adm)

I wanted to do a test and I went under Data Inputs > Files & Directories > New Local File & Directory > Browse > Var > Log the strange thing was that I can see half of the logs and half of the directories under there. All the directories and files that I can root:root and had other: r-- set permissions the file in question (test-combo.log) didn't have other:r-- permissions set. 

So why is splunk able to see files with these permissions

# file: vpn.log
# owner: root
# group: root


and not able to see files with this permission


# file: test-combo.log
# owner: syslog
# group: adm

is it because other is not set to read perms? What would be the significance of setting other to read?

Labels (1)
0 Karma


As the splunkd process runs with a user which is a member of the adm group, it should be able to read the file as such.

But remember that in order to "reach" the file you need to have access to the directories containing the file (it's not a Novell Netware where when the leaf access was propagated as need "upstream" ;-)).

The easiest way to verify the permissions would be to su to the splunk user and try to read the file with cat or less.

Also check your input status with

splunk list inputstatus

and see what splunk has to say about this file.

BTW, you don't have SELinux enabled, do you?

0 Karma

Super Champion

Yes, interesting.

Yes, I would check file reading with the cat command with splunk user first.

0 Karma

Super Champion

@splunktrainingu - You may encountering the Splunk issue.

Run Splunk with the least privileged access on Linux -

Please read the above idea description for details. Even though the idea status says "Under Point Threshold", I heard someone saying this has been resolved in Splunk 9.0.x. You can give it a try on a POC instance with the latest version of Splunk.


I hope this helps!!!

0 Karma

Super Champion

@splunktrainingu - Check below two things:

  • Make sure Splunk is running as the splunk user as you said.
    • ps -aux | grep "splunkd"
  • Check for error logs in splunkd.log files.
    • index=_internal source="*splunkd.log*" error


I hope this helps!!!

0 Karma


splunk is running as the splunk user

0 Karma


Hi @splunktrainingu 

yes, due to permission issue splunk is not able to read the file,

as splunk user comes under under other user, you need give read permissions for able to read 

also did did you see any permission related meesgaes in splunkd.log for test-combo.log source


Sanjay Reddy

If this reply helps you, Karma would be appreciated.

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.


0 Karma


Thank you I checked the splunkd.log and found out it doesn't have permissions but I already knew that. 

I am just trying to understand why? This doesn't make sense. Splunk user is part of ADM group, ADM group is applied to the file while can't splunk user read the file. What am I missing here? 


Insufficient permissions to read file='/var/log/test-combo.log' (hint: Permission denied , UID: 1001, GID: 1001).

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...