Getting Data In

Drop 9 of 10 Events (Sampling)

CMEOGNAD
Engager

Hi @ All Splunkynators

how to sample incoming (HEC) data?
I want get statistical data /events to save license volume, drop eg 9 of 10 of incoming events...

I look forward to your suggestions 🙂

Gegards - Markus

0 Karma

PickleRick
SplunkTrust
SplunkTrust

You can't do that directly in the splunk input phase. You can't keep "state" on event processing so you can't count events and keep track of which event you're processing at the moment. So there's no way to enforce the strictly "every 10th" rule. You can do somethig like time based as @rnowitzki suggested or try to implement some INGEST_EVAL logic to base your decision on random value (and check if it falls within some range) or - for example - message hash. But this will of course give you some percentage on average, not a strict 1 out of every N policy.

0 Karma

rnowitzki
Builder

Hi @CMEOGNAD ,

See my response to a similiar question.  You could drop events with seconds *1 to *9, keep *0

It will not be exactly 9 of 10. But with something like that you could randomly reduce the volume. 

Not sure if it makes sense with your data.

(or Cribl)


--
Karma and/or Solution tagging appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...