Here are some of the questions covered in the session:

Q1: How do you get logs from a Kubernetes Cluster?

Solution: 

• Set Log Collection in the wizard:

• See documentation - covers cases like host logs, multi-line logs, using pod annotations, sending events, etc.
• There are also many more tips about this topic from slides 8-14 in this deck

Documentation:

https://docs.splunk.com/observability/en/gdi/opentelemetry/collector-kubernetes/kubernetes-config-lo...

Q2: How can I optimize troubleshooting for K8s alerts?

Solution: 

• Utilize the Navigator links embedded in alerts 
• Autodetector alerts
• Utilize built-in metrics and dashboards
• Identify problematic nodes, pods, and containers using the hierarchy map in the Kubernetes Navigator
• Deploy the Splunk Distribution of the Otel Collector to your cluster for correlation of metrics, traces and logs (related content)
• Enrich telemetry data with custom metrics or adding relevant metadata to enhance troubleshooting 

Documentation:

• https://docs.splunk.com/observability/en/infrastructure/monitor/k8s-nav.html
• https://docs.splunk.com/observability/en/gdi/get-data-in/compute/k8s.html#get-started-k8s
• https://docs.splunk.com/observability/en/gdi/opentelemetry/collector-how-to.html#collector-how-to 
• https://docs.splunk.com/observability/en/metrics-and-metadata/relatedcontent.html

Q3: One of my containers produces events as json. Any pointers on how to teach the OTel Collector to read it in as json, I am getting multiple events strung together as one. Ideally is this something that I can achieve in the Splunk Otel helm chart?

Solution: 

• See details below on processing multi-line logs; this is likely why you are getting a multi-line json showing up as individual log lines

Documentation:

• https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/advanced-configuration.md#pro... 
• Installing with YAML manifests