Getting Data In

Splunk does not detect new files on CIFS-mounted directory until restart

koyachi
Explorer

Environment

  • Splunk Enterprise (single-instance: indexing + monitoring on same host)

  • OS: Linux

  • Log directory mounted via SMB/CIFS (smb2)

  • Input type: monitor://

  • Files are created by an external application on the CIFS share

Problem Description
Splunk does not automatically detect or ingest newly created log files under a CIFS-mounted directory when using a monitor:// input. Newly created files are only picked up after a manual Splunk restart. Existing files are indexed correctly at startup.

Observed Behavior

  • New log files are created in the monitored directory and are visible via ls

  • Files have correct permissions and ownership

  • Splunk does not ingest or log any monitor activity for new files while running

  • After restarting Splunk, all previously missed files are detected and indexed

Expected Behavior
Splunk should detect and ingest newly created files in the monitored directory without requiring a restart.

Key Finding
The monitored path is mounted via CIFS/SMB, and it appears that filesystem change notifications (inotify) are not being triggered for new files created on the share. As a result, Splunk is not notified of new files while running.

Troubleshooting Performed

  • Verified monitor stanza is loaded using splunk btool inputs list --debug

  • Verified file permissions and ownership (readable by Splunk user)

  • Tested with simple locally created files (echo > test.log)

  • Confirmed files appear in directory but are not indexed

  • Verified whitelist/blacklist regex does not exclude files

  • Adjusted CRC-related settings (crcSalt, initCrcLength)

  • Confirmed that polling_interval is not a valid or supported setting

  • Observed that Splunk only detects files during startup full directory scan

Can someone help to answer below so that it helps to understand if monitoring of files is possible for logs configured over mount.

  • Confirm whether monitor:// inputs are supported and expected to work reliably on CIFS/SMB-mounted directories

  • Clarify any documented or undocumented limitations regarding inotify on network-mounted filesystems

  • Advise on supported configurations or recommended alternatives (e.g., NFS, local landing directory, or forwarder-based architecture)

 

Labels (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

CIFS client in Linux is... well, something else. It has had its share of problems "since always" - it can hang on you when the source server rebooted and stuff like that.

Generally - network file systems have their fair share of possible issues but NFS4 seems more robust.

And the recommended way of ingesting files is of course deployment of the UF to the source server.

0 Karma
Get Updates on the Splunk Community!

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...

Purpose in Action: How Splunk Is Helping Power an Inclusive Future for All

At Cisco, purpose isn’t a tagline—it’s a commitment. Cisco’s FY25 Purpose Report outlines how the company is ...

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...