Platform: Data Management & Federation

Community Office Hours
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question

Platform: Data Management & Federation

Platform: Data Management & Federation

1 Comment
Cover Images - Office Hours (10).png
Published on ‎10-16-2025 03:16 PM by Splunk Employee | Updated on ‎11-24-2025 03:37 PM

Register here. This thread is for the Community Office Hours session on Platform: Data Management & Federation on Thurs, November 20, 2025 at 11am PT / 2pm ET. 

 

Ask the experts at Community Office Hours! An ongoing series where technical Splunk experts answer questions and provide how-to guidance on various Splunk product and use case topics.

 

What can I ask in this AMA?

  • How do Edge Processor/Ingest Processor and Federated Search for Amazon S3 work together? Can I get a demo?
  • How can I configure Edge Processor in Splunk Enterprise (on-prem)?
  • How can I onboard data from any data store or end points?
  • How does Splunk enable data federation across Amazon Security Lake (ASL) and S3? What tools are available to me?
  • How can I optimize my Edge Processor or Ingest Processor SPL2 pipelines?
  • Anything else you’d like to learn!

 

Please submit your questions at registration. You can also head to the #office-hours user Slack channel to ask questions (sign in with SSO here). 

 

Pre-submitted questions will be prioritized. After that, we will open the floor up to live Q&A with meeting participants.

 

Look forward to connecting!


Labels (2)
0 Karma
adepp
Splunk Employee
Splunk Employee
2 weeks ago
2 weeks ago

Hi everyone! Here are a few questions from the session (get the full Q&A deck and live recording in the #office-hours Slack channel) 

Q1: How can Edge Processor relieve ingestion work for a heavy forwarder?​

  • Too much data:​ noisy or verbose data sources​
  • Pre-Parsing:​ Event Breaking
  • Mask sensitive data​
  • Data Transformation & Enrichment

 

Q2: What are some use case for Edge Processor, Ingest Processor and Ingest Actions?​

  • Ingest Processor (Cloud)​

    • Data Admin/ SPL2 User​
    • Reduce noise/volume ​
    • Redact sensitive data ​
    • Send to indexes / s3​
    • Example: Cloud stack w/no infrastructure Masking and Anonymizing Data ​
  • Edge Processor (Cloud/On-Prem)​
    • Data Admin/ SPL2 User​
    • Enrich data via real-time threat detection w/KV Store lookups​
    • Modify raw events to remove fields and reduce storage ​
    • Convert complex data into metrics​
    • Route user events to a special index ​
    • Mask PII​

  • Ingest Actions (Cloud/On-Prem)​

    • Data Admin/ Props and Transform User​
    • Simple to medium use cases (simple routing) through a UI ​
    • Redact sensitive data with rulesets ​
    • Send to indexes / s3 / s3 Compatible ​
    • Masking PII ​

 

Q3: How to optimize Federated Search queries?​

  • ​FS-S3 DSU Optimization Best Practices:​
    • S3 Partitioning Tip: Partition data by time (e.g., year/month/day) and other common fields like sourcetype or host to minimize the amount of data scanned.​
    • Search Practices: Limit time ranges, specify federated index names, and use the partition attribute keys and right filters in the searches​
    • Monitoring & Troubleshooting: Use Job Inspector to identify bottlenecks, and the drill-down feature in the CMC license monitoring dashboard to find the top 10 apps, users, or searches for query-level optimization.​
  • Documentation: ​ Lantern article for partitioning best practices and guidelines

 

Other Questions (check the #office-hours Slack channel for responses):

  • What new capabilities are available across Data Management & Federation today?​
  • Federated Search for Amazon S3 vs data in Splunk. Which one is more taxing for scheduled alerts and Dashboards?​
  • I have difficulties ingesting https://github.com/splunk/botsv3 into cloud.​ I found journal.gz data files in rawdata folders of the archive, but when I use them to add data, parsing looks off.
  • I would like to understand how logs get parsed. There are so many different vendors. What schema Splunk uses
  • Is it possible to show a sample query to enrich on S3?
  • ​When setting up S3 integration, one has to map timestamp field in case one wants to correlate logs in splunk and logs in S3 bucket on time? If so, how is this correlation optimized if there is no index for S3 data? 
  • What are the indexes that are used for various types of logs?
0 Karma