Getting Data In

What the best strategy to discard all temporary data while testing on some forwarders?

dsmc_adv
Path Finder

We have a clustered environment that includes heavy forwarders, universal forwarders, and forwarders under Windows. The development team sometimes do performance tests and these generate a lot of data that we don't want to be indexed. We could add a new rule on the heavy forwarders to send to null queue all events during the tests , but can this be done at forwarder or universal forwarder level? Do you think that there is a better way to achieve this ?

Thank you

0 Karma

ddrillic
Ultra Champion

You can have the data indexed into specific indexes or add a specific field which indicates that this is a performance test data. Then it's easy to "simply" delete this type of data.

0 Karma

somesoni2
Revered Legend

Have a look at this Splunk documentation to know more about event routing and filter.
http://docs.splunk.com/Documentation/Splunk/6.4.0/Forwarding/Routeandfilterdatad#Filter_event_data_a...

The send to null queue can be done on universal forwarder if it's to be done without looking into individual events (purely based on index/source/sourcetype/host). If you need to look at the event data to filter, than you need to do routing/filtering in heavy forwarder/indexer

0 Karma

dsmc_adv
Path Finder

It looks like this only can be done at hf or indexer level as I suspected, but not in universal forwarder:

"Although similar to forwarder-based routing, queue routing can be performed by an indexer, as well as a heavy forwarder"

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...