Troubleshooting failure to process HTTP Event Collector events

[sbarnes_nj]

I currently use a C# client to log events via an SSL connection to HTTP Event Collector using Splunk Logging for .NET. Events are indexed as expected. Now a colleague is trying to build a Java client using Splunk Logging for Java and log4j to perform a similar job. Java events are not being indexed, and no errors appear at the server (splunkd.log) or on the Java end.

We can see through Wireshark that there is a persistent SSL connection from the Java client to Splunk. Occasionally packets are exchanged, but we cannot examine the contents. Aside from turning off SSL for everyone, is there any way to diagnose this better at the server end?

I have tried adjusting Splunk's Server Logging settings to DEBUG for several components, including HttpEventCollector, HttpListener, HttpInputEventParser, and HttpClientRequest. However, none of these have yielded any new log entries in "splunkd.log" about incoming HTTP Event Collector requests, or how they are being processed.

[xpac]

Did you try to setup a second HEC input with SSL disabled, just for testing? This way you could keep SSL for those that already work.

[sbarnes_nj]

As far as I can tell, SSL is a global setting that affects all HEC tokens. If you are talking about creating a second HEC that listens on a new port, I don't know how to do that.

[xpac]

I'm sorry, I literally assumed that this was possible as with almost all other inputs, but you're right.
Do you, by any chance, have an HF/SH available that's not yet running HEC that you could use for such a test?