User‑Driven Log Ingestion Control Framework

[prashanthan1987]

We often encounter situations where a particular application begins generating unusually high volumes of logs for a specific source, sourcetype, or index. This can quickly lead to excessive license consumption and unplanned cost overruns.

To address this, I am exploring an approach that provides more direct control to application owners or stakeholders—allowing them to decide whether log ingestion should be temporarily paused or approved for suspension. This could be achieved through an automated workflow where users receive an API-triggered notification or an email containing a decision link. With this model, we can prevent unnecessary license impact while ensuring that any action taken has explicit user consent.

I’d appreciate your thoughts on this approach.

[livehybrid]

Hi @prashanthan1987 

You could look at using Ingest Actions API endpoints to leverage this, this will allow you to make changes to drop certain events based on your criteria programatically using automation, but I would recommend heavily testing this first and having plenty of guardrails to ensure nobody can manipulate it to drop things that shouldnt be dropped!

Check out https://help.splunk.com/en/splunk-enterprise/leverage-rest-apis/rest-api-reference/10.0/input-endpoi... and specifically the '/services/data/ingest/rulesets/' endpoint, such as:

curl -k https://HOST:PORT/services/data/ingest/rulesets\?output_mode=json -d name=RULESET_NAME -d sourcetype=SOURCETYPE -d rules="[
          {
            \"name\": \"RULE_NAME\",
            \"action\": \"route\",
            \"clone\": false,
            \"dest\": \"DESTINATION\",
            \"cond\": {
              \"type\": \"eval\",
              \"expr\": \"someExpressionHere\"
            }
          }
        ]"

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

Yes, but regardless of whether you use transforms, rulesets, ingest actions, edge processor, it's still something based on strict rules you have to set prior to ingesting data. There is no "hold and decide" mechanics.

[PickleRick]

It doesn't work like this. Even if you made something that would selectively enable and disable HEC tokens and set specific sources to use specific tokens, there is no way to "hold" any specific events and "accept" or "reject" them within just Splunk's framework alone. You'd need to create an intermediate solution providing such functionality before ingesting the data into Splunk.