Splunk Edge Processor | Popular Use Cases to Get Started with Edge Processor

[adepp]

Splunk Edge Processor offers more efficient, flexible data transformation – helping you reduce noise, control costs, and gain visibility and control over your data in motion. It works at the edge of your network and is included with your Splunk Cloud Platform, available at no additional cost. Learn more about the Edge Processor solution, including resources to get started. 

With Edge Processor, you can:

• Filter low-value or noisy data, like DEBUG logs. 
• Enrich and extract only the critical data.
• Route different “slices” of data to Splunk platform and Amazon S3.

Edge Processor pipelines use SPL2 to define the logic for filtering, masking, and transforming data before routing it to supported destinations. SPL2 lets you use one common language to both search and transform your data. This gives you the new flexibility to filter out parts of the event itself, in addition to the entire event. Today, Splunk Edge Processor can receive data from many sources, including Universal Forwarders, HTTP Event Collector, syslog and more.

Use Case Prerequisites 

Before you can implement use cases with Edge Processor, make sure you have:

1. Connected your Edge Processor tenant to your Splunk Cloud Platform deployment via the first-time setup instructions. 
2. Created an Edge Processor instance by following the steps under “configure and deploy Edge Processor”.

Splunk Edge Processor Common Use Cases 

The links below walk you through common use cases that Splunk Edge Processor can address. These can help you reduce ingest volume to optimize costs around data storage and transfer, protect sensitive information, and significantly improve your time to value. 

Filter and Route Data

Reduce and route logs for cost-effective storage [Blog]

Step-by-step guidance to reduce substantial volumes of ingested logs and route them to Amazon S3 for cost-effective storage.  

Filter Kubernetes data over HTTP Event Collector (HEC) [Video]

This video walks you through how to build a pipeline to filter noisy events from Kubernetes pods using the HTTP Event Collector (HEC). 

Reduce security firewall logs (PAN and Cisco) with Splunk Edge Processor [Lantern]

Are you swamped by the relentless surge of log data from your Palo Alto Networks (PAN) and Cisco devices? Follow this step-by-step guidance to reduce your firewall logs with Edge Processor. You can also watch the demo video walkthrough or read the blog for more context. 

Filter verbose data sources and transform content for Windows system events [Blog]

Scroll down this blog to see how to filter verbose data sources, such as Windows event logs, and to retain selected events or content within an event. Then route an unfiltered copy to AWS S3 bucket.

Transform, Mask, and Route Data

Enrich data via real-time threat detection with KV Store lookups [Lantern]

By creating and applying a pipeline that uses a lookup, you can configure an Edge Processor to add more information to the received data before sending that data to a destination (docs). In this case, our objective is to use the event fields present in your ingested data to preemptively identify and flag malicious activity. 

Modify raw events to remove fields and reduce storage [Video]

Remove unwanted fields from a raw event and reconstruct it with a reduced number of fields to optimize storage in the Splunk platform. Similar logic can be used to drop as many fields as desired to reduce your storage footprint and improve performance.

Convert complex data into metrics with Edge Processor [Lantern]

This step-by-step guide walks you through how to transform complex bloated data into metrics by pre-processing your data with Edge Processor so you can cut storage costs. For a simplified version of this process, see Converting logs into metrics with Edge Processor for beginners.

Route root user events to a special index [Lantern]

This use case provides step-by-step guidance to filter any events relating to the “root” user in your Linux authentication data and send them to an index they’ve created for you called admin.

Mask sensitive credit card information [Video]

Masking logic can be applied on credit card information to extract the card number field and replace the value with a string of your choosing, ensuring that the data remains secure and your business complies with data privacy regulations.

Mask IP addresses from a specific range [Lantern]

There are multiple ways of achieving this IP masking use case with SPL2, depending on how flexible you want your pipeline to be. This article looks at two possible methods 1) using eval replace and 2) using rex and cidrmatch.

Additional Resources 

Check out these additional resources to learn more and get started using Edge Processor: 

• Edge Processor Resource Hub
• Lantern step-by-step ‘Getting Started’ guide
• Requirements to use Edge Processor and how to request access if you do not already have it
• Splunk Edge Processor release notes and documentation