Can I restrict the log ingestion when the index capacity reaches its limit on per day basis?

[mala_splunk_91]

Hi, 

In Splunk cloud, Can I restrict the log ingestion when the index capacity reaches its limit on per day basis?

I have logs which is exceeding its indexing capacity on certain days. Is there any way I can block ingestion if the capacity reaches its threshold?

Also, I have another question, Is it possible for me to edit the configuration files to filter logs or send it null queue on the Splunk cloud?

If I want to create custom app to do so. Please share me any related documents to follow.

Thanks, 

Mala Sundaramoorthy

[gcusello]

Hi @mala_splunk_91,

as @VatsalJagani said, there isn't any automatic way to do this.

Obviously you can create an alert that fires when you're reaching e.g. the 50% at midday or the 80% at 5 PM.

So you can turn off some input when the alert fires, but not automatically.

Maybe it's  possible having Phantom, but I never tried.

about configurations, you can modify them only by interface on Splunk Cloud.

It's easier if you have to take on-premise logs using Forwarders, but anyway, always in manual mode not automatically.

About the way to create a custom App, it's a very easy App:

• it must be deployed on all the Heavy Forwarders you have, as you know it's a best practice to have at least two HFs as Concentrators to take all the traffic from on-premise systems and send it to Splunk Cloud;
• it must contain at list two files:
  • props.conf,
  • transforms.conf
• you can find the way to filter logs at https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Forwarding/Routeandfilterdatad#Filter_eve...
• You can deploy it to your HFs manually or using the Deployment Server;
• you can check your app using the Add-On Builder App https://splunkbase.splunk.com/app/2962/ but I neved did it because this TA is very simple.

It will be easier when the data Stream Processor will be available (https://docs.splunk.com/Documentation/DSP/1.3.0/User/Filter).

Ciao.

Giuseppe

[VatsalJagani]

@mala_splunk_91 

• I have logs which is exceeding its indexing capacity on certain days. Is there any way I can block ingestion if the capacity reaches its threshold?
  • Currently, there is no way Splunk has built in to do so.
• Also, I have another question, Is it possible for me to edit the configuration files to filter logs or send it null queue on the Splunk cloud?
  • You guessed it right, you will require to deploy those configurations in a custom App.
  • How to install App on Splunk Cloud - https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Admin/PrivateApps
  • How to do NullQueue (example) - https://community.splunk.com/t5/Getting-Data-In/Filtering-events-using-NullQueue/m-p/66392 

I hope this helps!!!