Splunk not ingesting some log files

[user789]

I have set splunk to ingest the /var/log directory. On this particular host, I go to filter by "source", and only see 2 sources:

• /var/log/messages
• /var/log/maillog

Why is it not seeing other files and folders? For example, there is /var/log/audit/audit.log.

[richgalloway]

Does the account running Splunk have read access to the missing files? Often, files in /var/log are secured so only root can read them.

[user789]

Splunk is running as "sudo", so this should be fine, right?

[richgalloway]

"fine" as in able to read the files, yes. "fine" as in a good way to run Splunk, no. Running Splunk (or any non-OS process) as root increases your attack surface.

[user789]

So splunk is able to read these files and into directories, yet they are not populating on the splunk server.

[richgalloway]

Search index=_internal to verify the forwarder is sending data to the indexers. Verify you are looking in the right index for the data.

[user789]

I ran the query for index=_internal, and I do not see any of my hosts showing up.