Autoscaling Kubernetes Workloads with Splunk

[clemoine]

About

What if you had the ability to scale your Kubernetes pods as a result of information output from Splunk? I recently had this idea and thought it would be exciting to provide customers with such a capability to see what amazing solutions you could come up with. As a result, we at Splunk recently contributed a new integration to the well known CNCF (Cloud Native Computing Foundation) graduated project, KEDA (Kubernetes Event Driven Autoscaling). KEDA fills gaps in Kubernetes horizontal pod autoscaling by supporting the scaling of pods from a variety of different data sources such as message queues, time series data, application performance metrics, etc.

Getting started

With v2.15 of KEDA, you can now autoscale your Kubernetes workloads by configuring the new Splunk scaler to utilize data from a saved search. The integration is relatively simple: you create a ScaledObject resource to tell KEDA which deployment to scale, by how much, and under what conditions. In the example below, we have a saved search in our Splunk instance called "my-saved-search" and we tell KEDA to scale our nginx deployment if the field "count" from our search has a value greater than or equal to 10. Once our "count" field reaches less than 10, our workload is then scaled back down. By getting creative with the configuration below, you can scale on-demand to reduce costs and increase performance by using data you already have in Splunk!

---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: splunk-scaledobject
spec:
  minReplicaCount: 1
  maxReplicaCount: 3
  scaleTargetRef:
    name: nginx
  triggers:
  - type: splunk
    authenticationRef:
      name: splunk-auth
    metadata:
      host: https://splunk.default.svc.cluster.local:8089
      targetValue: "1"
      activationValue: "10"
      savedSearchName: my-saved-search
      valueField: count

For more detailed information, please see the KEDA documentation for the Splunk scaler.

Support

Have an idea for a new feature for the Splunk scaler? Feel free to raise an issue in KEDA's GitHub project or even better, raise a pull request to contribute!

Credits

This effort was successful thanks to collaboration with the KEDA project maintainers. We couldn't have done it without them and appreciate all of their valuable input along the way.