I am observing an intermittent exception "System.Net.WebException - The operation has timed out" occurring within a C# Splunk SDK program I have deployed as a CLR stored procedure within SQL Server 2008 R2. For background, I have created a class with two methods:
InitializeSplunkConnection (patterned after the examples/authenticate solution)
SubmitEventToSplunk (patterned after the examples/submit solution)
The exception can occur within either method. The class is included below. I am providing credentials via a .splunkrc file. I've verified that these credentials can be used to authenticate against the splunkd admin interface on 8089. This code had been working reliably for some time during testing and only recently began throwing the exception. I have verified that nothing has changed in either the Splunk instance or the machine running SQL Server. Restarting SQL Server will generally fix the problem temporarily, perhaps allowing for a successful invocation or two of either method, but then the timeout problem returns.
As I stated, I'm deploying each of the methods in this class as a stored procedure (via the SQL statements required to first create all necessary assemblies and then the procedures):
I'm currently using Splunk SDK version 1, downloaded ~July 2013. I see there is a 1.0.1 SDK version now. I'm wondering if there have been any fixes implemented there that might benefit me. Thanks for any assistance.
Some additional information about the problem I am observing. After restarting my MSSQL service (presumably resetting some aspect of my network connection to Splunk?), I am able to make multiple manual invocations the initialize connection procedure as:
Each manual invocation above behaves normally (e.g. returning almost instantaneously and yielding the message "Splunk instance initialized for instance specified in "+pathToDotSplunkRCFile). If I then invoke the third stored procedure referenced above, seeking to do a bulk record-at-a-time insert of data to a splunk index, the procedure will end with a timeout on what appears to consistently be the 101st invocation of the procedure sp_SubmitEventToSplunk (i.e. I observer 100 records successfully added to the index before the procedure ends with the timeout error). I've repeated this experiment a number of times, and each time I observe 100 records added before the timeout. I'm wondering if this behavior might be resulting from a Splunk configuration parameter.
public class Program
public static Command cs_cli;
public static void InitializeSplunkConnection(String parameterFilePath)
s_cli = new Command("plugin").Init();
// Test connection
SqlContext.Pipe.Send("Splunk instance initialized for instance specified in "+parameterFilePath);
public static void SubmitEventToSplunk(String splunkIndex, String splunkSource, String splunkSourceType, String eventMessage)
if (s_cli == null)
throw new Exception("Splunk connection must be initialized via InitializeSplunkConnection().");
var service = Service.Connect(s_cli.Opts);
var args = new ReceiverSubmitArgs
Source = splunkSource,
SourceType = splunkSourceType,
Index = splunkIndex,
var receiver = new Receiver(service);
// Submit to specified index
Your issue doesn't seem to be related to Splunk configuration parameters. Would it be possible for you to reproduce the problem outside of SQL server? If you send the program and some test data I would be happy to look at it.