Platform: Data Management & Federation

adepp — Mon, 23 Mar 2026 19:57:11 GMT

Register here. This thread is for the Community Office Hours session on Platform: Data Management & Federation on Thurs, November 20, 2025 at 11am PT / 2pm ET.

Ask the experts at Community Office Hours! An ongoing series where technical Splunk experts answer questions and provide how-to guidance on various Splunk product and use case topics.

What can I ask in this AMA?

How do Edge Processor/Ingest Processor and Federated Search for Amazon S3 work together? Can I get a demo?
How can I configure Edge Processor in Splunk Enterprise (on-prem)?
How can I onboard data from any data store or end points?
How does Splunk enable data federation across Amazon Security Lake (ASL) and S3? What tools are available to me?
How can I optimize my Edge Processor or Ingest Processor SPL2 pipelines?
Anything else you’d like to learn!

Please submit your questions at registration. You can also head to the #office-hours user Slack channel to ask questions (sign in with SSO here).

Pre-submitted questions will be prioritized. After that, we will open the floor up to live Q&A with meeting participants.

Look forward to connecting!

Re: Platform: Data Management & Federation

adepp — Mon, 24 Nov 2025 23:37:02 GMT

Hi everyone! Here are a few questions from the session (get the full Q&A deck and live recording in the #office-hours Slack channel)

Q1: How can Edge Processor relieve ingestion work for a heavy forwarder?

Too much data: noisy or verbose data sources
Pre-Parsing: Event Breaking
Mask sensitive data
Data Transformation & Enrichment

Q2: What are some use case for Edge Processor, Ingest Processor and Ingest Actions?

Ingest Processor (Cloud)
- Data Admin/ SPL2 User
- Reduce noise/volume
- Redact sensitive data
- Send to indexes / s3
- Example: Cloud stack w/no infrastructure Masking and Anonymizing Data
Edge Processor (Cloud/On-Prem)
- Data Admin/ SPL2 User
- Enrich data via real-time threat detection w/KV Store lookups
- Modify raw events to remove fields and reduce storage
- Convert complex data into metrics
- Route user events to a special index
- Mask PII
Ingest Actions (Cloud/On-Prem)
- Data Admin/ Props and Transform User
- Simple to medium use cases (simple routing) through a UI
- Redact sensitive data with rulesets
- Send to indexes / s3 / s3 Compatible
- Masking PII

Q3: How to optimize Federated Search queries?

FS-S3 DSU Optimization Best Practices:
- S3 Partitioning Tip: Partition data by time (e.g., year/month/day) and other common fields like sourcetype or host to minimize the amount of data scanned.
- Search Practices: Limit time ranges, specify federated index names, and use the partition attribute keys and right filters in the searches
- Monitoring & Troubleshooting: Use Job Inspector to identify bottlenecks, and the drill-down feature in the CMC license monitoring dashboard to find the top 10 apps, users, or searches for query-level optimization.
Documentation: Lantern article for partitioning best practices and guidelines

Other Questions (check the #office-hours Slack channel for responses):

What new capabilities are available across Data Management & Federation today?
Federated Search for Amazon S3 vs data in Splunk. Which one is more taxing for scheduled alerts and Dashboards?
I have difficulties ingesting https://github.com/splunk/botsv3 into cloud. I found journal.gz data files in rawdata folders of the archive, but when I use them to add data, parsing looks off.
I would like to understand how logs get parsed. There are so many different vendors. What schema Splunk uses
Is it possible to show a sample query to enrich on S3?
When setting up S3 integration, one has to map timestamp field in case one wants to correlate logs in splunk and logs in S3 bucket on time? If so, how is this correlation optimized if there is no index for S3 data?
What are the indexes that are used for various types of logs?

rss.livelink.thread@place:occasion

Platform: Data Management & Federation

Re: Platform: Data Management & Federation