Monitoring Splunk

How to monitor Application in OpenShift Cluster with Splunk?

noyog
Engager

Hi,

My application is running on OpenShift pods.
The application accepts API calls on port 9443.
In front of the pods, we have ELB which is the public entry point into the application.
The ELB accepts traffic on port 443 and pass them through to the pods over 9443 (it doesn't terminate TLS).

My Splunk server is external to the OpenShift cluster.
We created a Splunk add-on that performs API calls to the application (it works when the application is installed on VM).

When we try to monitor the application behind OpenShift, Splunk gets a 503 (Application is not available) response from OpenShift.
If we do an API call via CURL command from the Splunk server, the call is successful so we know the routing from the ELB to the OpenShift pods works.
Interestingly, when we change the URL in the CURL command to be one of the ELB IPs, we reproduce the issue we see in Splunk.

Is there any special configuration we need to perform in order for the monitoring to work against OpenShift?

0 Karma
1 Solution

noyog
Engager

We managed to resolve the issue.
Our add-on was built using Splunk's add-on builder and that comes with httplib2 v0.9.2.
That version doesn't support SNI and as a result, the API calls from the add-on to the OpenShift cluster were answered with 503 ('The host doesn't exist' error).
It seems like OpenShift requires SNI so the pods know how to direct the calls.

We manually updated the httplib2 version in the add-on and it resolved the issue.
Any httplib2 version above 0.9.2 supports SNI.

CURL supports SNI which is why those calls worked.

View solution in original post

0 Karma

noyog
Engager

We managed to resolve the issue.
Our add-on was built using Splunk's add-on builder and that comes with httplib2 v0.9.2.
That version doesn't support SNI and as a result, the API calls from the add-on to the OpenShift cluster were answered with 503 ('The host doesn't exist' error).
It seems like OpenShift requires SNI so the pods know how to direct the calls.

We manually updated the httplib2 version in the add-on and it resolved the issue.
Any httplib2 version above 0.9.2 supports SNI.

CURL supports SNI which is why those calls worked.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...