Data Management in Observability Cloud

Community Office Hours

Data Management in Observability Cloud

1 Comment
Cover Images - Office Hours (15).png
Published on ‎03-13-2025 12:26 PM by Splunk Employee | Updated on ‎08-08-2025 01:29 PM

Watch On-DemandThis thread is for the Community Office Hours session on Data Management in Observability Cloud on Tues, May 20, 2025 at 1pm PT / 4pm ET

 

Ask the experts at Community Office Hours! An ongoing series where technical Splunk experts answer questions and provide how-to guidance on various Splunk product and use case topics.

 

What can I ask in this AMA?

- What capabilities does Splunk have to balance costs and data volume?
- How does the Splunk Distribution of the OpenTelemetry Collector help filter out data to reduce ingestion costs?
- What are some ways that I can scale confidently without breaking the bank?
- How can I filter, aggregate, and archive data for optimal storage and analytics?
- Anything else you'd like to learn about!

 

Please submit your questions at registration. You can also head to the #office-hours user Slack channel to ask questions (request access here)

 

Pre-submitted questions will be prioritized. After that, we will open the floor up to live Q&A with meeting participants.

 

Look forward to connecting!



Labels (2)
0 Karma
ArifV
Splunk Employee

Hello! Here are the recap materials from the session:

Here are the questions that we received in the session (more detailed solutions and info can be found in the slide deck)

Q1What is the best strategy to reduce ingestion costs? is there a way to show how much is being reduced? 

A: 

  • Archive all Unused Metrics
  • Figure out lowly used metrics
  • Sign up for Automated Archiving to Automate

Documentation:

Q2: What are the best recommendations relating to data retention in Splunk, or even cold restore?

A: 

Metrics

  • 1 Second Resolution: 3 Months
  • 10 Seconds or more: 13 Months

APM

  • Raw Traces: 8 Days
  • Traces of interest viewed in the Splunk APM user interface: up to 13 Months
  • Profiling Data: 8 Days

RUM

  • Spans: 8 Days
  • Session Replay: 8 Days

Synthetics

  • Run Results: 8 Days

Metric Data: 13 Months

Logs

  • Varies by industry, regulations, source, and organization policies. See the Splunk Lantern article for more details and recommendations.

Other

Documentation:

Q3: How would I manage log ingestion made by fluent like we do with Splunk Universal Forwarder and props?

A: The Fluent Forward receiver allows the Splunk Distribution of the OpenTelemetry Collector to collect events using the bundled Fluentd application. The receiver accepts data formatted as Fluent Forward events through a TCP connection. All three Fluent event types, message, forward, and packed forward, are supported, including compressed packed forward. 

However, this integration will be deprecated in October of 2025 as we have shifted to native OpenTelemetry log collection.

Our best practice recommendation is as follows:

  • Kubernetes: OpenTelemetry native log collection
  • Stand-alone Linux/Windows hosts: Universal Forwarder log collection

Documentation: