Modular inputs for Microsoft cloud services Add-on fill up the disk space???

[captcha]

I am using Microsoft Cloud services Add-on to ingest the logs from Azure storage account. The modular inputs for the app does not have clean up mechanism, filling up the disk space and causing service interruptions. Are others using this app see the same issue? Do you have a batch jobs to clean up the files?

[Rhidian]

Did you ever resolve this issue?

[francoisternois]

Hi,

I'm facing the same issue. This folder (splunk/var/lib/splunk/modinputs/mscs_storage_blob) keeps growing with checkpoint files, especially on few storage account folders. Each file referencing the name of the storage account file that Splunk ingest.

I guess that it check all the files to compare with the files on the storage account (explaining performance issue) trying to find a new file to ingest.

Did you find a way to solve this issue ?

[MuS]

Hi captcha,

you can create a $SPLUNK_HOME/etc/log-local.cfg and add a log rotation configuration for that log file. Use the $SPLUNK_HOME/etc/log.cfg file as example. Find some basic information about the log management process here https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/WhatSplunklogsaboutitself#The_lo...

Hope that helps ...

cheers, MuS

[MuS]

In the log.cfg search for the appender.xyz.maxFileSize or appender.xyz.maxBackupIndex options for some examples.

cheers, MuS

[hkubavat_splunk]

I think Captcha is facing issue of Disk space because of files that is created from modular inputs and not the log files of modular input. Captcha can you please confirm?

[captcha]

Hello MuS and hkubavat,

I think I did not state my questions correctly. Looks like the modular inputs creates a checkpoint file, this file grows really big causing some disk space issues. Deleting these checkpoint files will initiate reindexing of the logs. Any suggestions on how to deal with this issue?

[MuS]

Okay to me that sounds like a bug in the mscs_checkpointer.py script, because you should only have the latest checkpoint in that file and not a list of checkpoints. Haven't checked very deeply but it sounds like it does an append instead of a replace in the checkpoint file.

Hope this helps ...

cheers, MuS

[hkubavat_splunk]

There is a limitation in API of azure where you cannot filter the logs as because of that you have to check all the files. In MSCS for one blob, there is one checkpoint file. So my question to Captcha is you are facing an issue is because of disk space or IO Read/write operation? It may be a case your CPU usage is normal though your system is slow.