Here are a few questions from the session (get the full Q&A deck and live recording in the #office-hours Slack channel):
Q1: What are the optimal forwarder configurations forgetting data into Splunk Cloud with Forwarders (e.g., best practices architectures, with a more technical how to)
- Use Universal Forwarders, when possible (unless an HF is necessary - for transforms, routing, or to run some apps such as DB Connect)
- Include EVENT_BREAKER in props.conf
- Avoid intermediate forwarders
- Layers add complexity, increase failure points, and may lead to uneven distribution of events across indexers.
- Use volume-based load balancing over time-based load balancing
- Consider using HEC
- Documentation: Splunk Validated Architectures
Q2: What are some new features/solutions for getting data into Splunk?
- Edge Processor
- Available to all, but must also be a Splunk Cloud customer
- Processes data locally, before it is sent to indexers
- Ingest Processor
- Available in Splunk Cloud only
- Processes data in Splunk Cloud, before shipping to destination(s)
- Ingest Actions
- Available to all
- Unlike transforms, can process cooked data
- OpenTelemetry Collector (CNCF, second largest project after Kubernetes)
-
- Open source
- Supports HEC, OTLP
Documentation:
Q3: I am very new to Edge and Ingest Processor. Could you please walk-through with an example from a beginner's perspective?
Resources:
Documentation:
Other Questions/Topics (check the #office-hours Slack channel for responses):
- How can I start routing and filtering with the ingest processor after it has been enabled in the Cloud Stack?
- Splunk licence saving tips
- AWS S3 data ingestion through SQS
- Let's talk about Hybrid landscape
- How to efficiently GDI Commvault storage data?
- How to efficiently GDI Dell Unity data?
- How to efficiently GDI Microsoft System Center VM Manager (SCVMM) and System Center Operations Manager (SCOM) data?
- Best services to use for best practice for S3 data lakes and Splunk? Security, anomaly detection, scanning of logs, etc.
- GDI and Auto-discovery of entities from BMC Helix/ADMM and auto creation and dependency mapping of related services, entities in ITSI
- Testing custom apps with splunk cloud. IE testing in prod because it’s cloud.
- I am sending data via HEC to a stand alone Indexer, how can I send data via HEC to an Indexer Cluster instead? Best practice ?
- When Edge processor and Ingest Processor will be available for Splunk onprem ?
- I have an indexer cluster where an IDX is getting more activity that others? how can i fix it ?