Splunk Observability Cloud

Do you have any ideas for checks on traces Java-applications in the Kubernetes?

KirillYanovsky
New Member

Hi, all

I took a trial for 14 days, set up integration with Kubernetes (EKS) and tried to configure 4 services (of the same type, Java) to send traces.
Installed Helm-chart with splunk-otel-collector + integrated splunk-otel-javaagent.jar (all according to the instructions in the section Integration)

All services have the same configuration. 
But 1 service send traces, other 3 - not. 
(BTW, early the same 4 services in DataDog give traces, without problems)
There are no errors in the logs similar to these:
https://github.com/signalfx/splunk-otel-java/blob/main/docs/troubleshooting.md

Have any ideas for checks ? 

Thanks

Labels (1)
0 Karma

jshawatsplunk
Splunk Employee
Splunk Employee

I realize your trial probably expired if not extended. Did you get traces working? 

Splunk tracing works a little differently than some APM agents in that it only sends traces if there are requests executed.  So each JVM would have had to process java requests (active traffic).  That is one aspect - making sure requests are hitting each JVM. 

I would also check to make sure there was not any port block/conflict from the node/pods where you did not see traces from reaching the agent collector. If the collector is running, it should detect the local JVMs are instrument automatically.  But if you're not seeing that, like tail java to look for javaagent, then you can also simply add the javaagent reference. 

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...