Submitting events to an index: even when no errors are returned, not all events are found on Splunk.

[paulomenezes]

Hello, guys!

Well, I'm trying to send data to a Splunk index via Splunk SDK for NodeJS.

The thing is: randomly, I was receiving this error for some events:
Error: connect ECONNRESET 127.0.0.1:8089 at TCPConnectWrap.afterConnect [as oncomplete]

Ok, then I implemented a retrial mechanism for that error. It seemed to work, since I could send much more data to Splunk than ever. But, still, not all data are sent. And worst, no error is returned on callback.
I thought it could be concurrency, then I created a submission pool which only submits 500 events max (I even tried with 10 lol) concurrently. Nothing changed.

Out of 32k records, only between 25k and 28k are properly sent to Splunk.

Have you guys ever been through this situation?
Thanks in advance.

[bjoernjensen]

Hey,

"ECONNRESET" points out, that the receiving side rudely closed its end of the TCP connection.

Could the index be overloaded and simply kills the connection as a result? Are there any hints in your splunkd.log?

Cheerz,
Björn

[paulomenezes]

Hey, Bjorn!
Well, nothing is logged on splunkd.log when the error is raised.

Could this error happen due to too many data being sent to the index in a short period of time? I'll try to give some time to the index to breathe between batches' execution.

Thanks!