I have a CIFS mount from Azure on a server.
Then a Splunk forwarder monitoring the mounted folder.
I discovered that Splunk can detect the files when starting, but not later when a file is modified.
Explanation :
Folder modification time in MAFS (Microsoft Azure File System) is not updated ! Splunk is unable to properly monitor the folder as there's no change triggering ingestion of the new files. This is not a bug in Splunk, but limitation of the Azure File Storage ... even windows explorer and Azure web interface are showing creation time as the last modification date !
Full list of limitations can be found here:
https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/Features-Not-Supported-By-the...
The possible workarounds are :
manually update your file modification time, to force detection.
The only workaround we were able to come up with (that actually works) was to update the destination folder last modification time manually
(e.g. by using a script after uploading log files):
PowerShell
(Get-Item
Or not monitor Azure, and copy the files outside of the mount each time.
For linux systems you can execute touch on the folder that the logs reside in.
Location of files: /mnt/mymountedazurefile/myfolder/logfile.json
Mounted shared drive: /mnt/mymountedazurefile
Command: touch /mnt/mymountedazurefile/myfolder
Splunk detects the new files instantly. It is a better solution than restarting splunk service.
We have a splunk server that has a mounted shared from Azure Files. All the app services in azure write to the same shared disk for log files.
This is a temporary solution until we migrate application logging to Azure Event Hub.
A useful commad for debuggin:
/opt/splunk/bin/splunk list inputstatus | grep <filename>
Explanation :
Folder modification time in MAFS (Microsoft Azure File System) is not updated ! Splunk is unable to properly monitor the folder as there's no change triggering ingestion of the new files. This is not a bug in Splunk, but limitation of the Azure File Storage ... even windows explorer and Azure web interface are showing creation time as the last modification date !
Full list of limitations can be found here:
https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/Features-Not-Supported-By-the...
The possible workarounds are :
manually update your file modification time, to force detection.
The only workaround we were able to come up with (that actually works) was to update the destination folder last modification time manually
(e.g. by using a script after uploading log files):
PowerShell
(Get-Item
Or not monitor Azure, and copy the files outside of the mount each time.