topic Re: Push Splunk events from .NET application in All Apps and Add-ons

Push Splunk events from .NET application

Josh — Sun, 22 Aug 2010 16:03:34 GMT

Hello All,

We would like to be able to push Splunk event from a .NET application. There is a JAVA rest API to do the same:

http://code.google.com/p/splunk-java-sdk/wiki/UploadingEvents

Does anyone know of an API available to do this with .NET, also I believe this could be achieved using UDP/TCP push from .NET has anyone attempted this? Any examples?

Thanks, Josh

Re: Push Splunk events from .NET application

gkanapathy — Sun, 22 Aug 2010 22:32:15 GMT

You could just use log4net, or write to a file, or simply open a network socket and send the text over it.

Re: Push Splunk events from .NET application

Josh — Mon, 23 Aug 2010 07:05:11 GMT

Hey guys I figured it out by looking at the Java source code from google.

You can just post an event in .NET using the REST API by accessing the

/services/receivers/simple or /services/receivers/stream endpoints.

An simple example for anyone who is interested:

using System;
using System.IO;
using System.Net;
using System.Xml;
using System.Web;

namespace SplunkNetClient
{
    class SplunkTestClient
    {

        static XmlDocument executeRequest(string url, string sessionKey, string args, string httpMethod)
        {
            ServicePointManager.ServerCertificateValidationCallback += delegate { return true; };

            HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);
            req.Method = httpMethod;
            if (!string.IsNullOrEmpty(sessionKey))
                req.Headers.Add("Authorization", "Splunk " + sessionKey);

            if (httpMethod == "POST")
            {
                if (!string.IsNullOrEmpty(args)) req.ContentLength = args.Length;
                req.ContentType = "application/x-www-form-urlencoded";
                Stream reqStream = req.GetRequestStream();
                StreamWriter sw = new StreamWriter(reqStream);
                sw.Write(args);
                sw.Close();
            }

            HttpWebResponse res = (HttpWebResponse)req.GetResponse();
            StreamReader sr = new StreamReader(res.GetResponseStream());
            if (sr.EndOfStream) return null;
            XmlDocument xmlDoc = new XmlDocument();
            xmlDoc.Load(sr);
            sr.Close();
            return xmlDoc;
        }


        static void Main(string[] args)
        {
            string baseurl = "https://localhost:8089";
            string user = "admin";
            string password = "changeme";
            string searchQuery = "host=josh";
            string index = "main";

            // Authenticate on the server and get the session key
            //
            string url = baseurl + "/services/auth/login";
            string reqArgs = string.Format("username={0}&password={1}", user, password);
            XmlDocument doc = executeRequest(url, null, reqArgs, "POST");
            string sessionKey = doc.SelectSingleNode("/response/sessionKey").InnerText;
            Console.WriteLine("Session key: " + sessionKey);

            //Upload an event to splunk
            url = baseurl + "/services/receivers/simple?index=main&sourcetype=wsdl&source=EDiscovery&host=josh";
            reqArgs = string.Format("env={0},url={1},response={2},app={3},version={4}", "\"EDEV\"","\"test\"", "\"200\"","\"dpsapap\"","\"1.0.7.5467\"");
            doc = executeRequest(url, sessionKey, reqArgs, "POST");


            // Dispatch a new search and return search id
            //
            url = baseurl + "/services/search/jobs";
            reqArgs = string.Format("search=search {0}", HttpUtility.UrlPathEncode(searchQuery));
            doc = executeRequest(url, sessionKey, reqArgs, "POST");
            string sid = doc.SelectSingleNode("/response/sid").InnerText;
            Console.WriteLine("Search id: " + sid);

            // Wait for search to finish
            //
            url = baseurl + "/services/search/jobs/" + sid;
            bool isDone = false;
            int eventCount = 0;
            do
            {
                doc = executeRequest(url, sessionKey, null, "GET");
                if (doc == null)
                {
                    System.Threading.Thread.Sleep(200);
                    continue;
                }
                XmlNamespaceManager context = new XmlNamespaceManager(doc.NameTable);
                context.AddNamespace("s", "http://dev.splunk.com/ns/rest");
                context.AddNamespace("feed", "http://www.w3.org/2005/Atom");
                XmlNode ecNode = doc.SelectSingleNode("//feed:entry/feed:content/s:dict/s:key[@name='eventCount'][1]", context);
                eventCount = int.Parse(ecNode.InnerText);
                XmlNode idNode = doc.SelectSingleNode("//feed:entry/feed:content/s:dict/s:key[@name='isDone'][1]", context);
                isDone = idNode.InnerText == "1" ? true : false;
            } while (!isDone);
            Console.WriteLine(string.Format("Search job is done, {0} results found", eventCount));

            // Get search results
            //
            if (eventCount > 0)
            {
                url = string.Format("{0}/services/search/jobs/{1}/results", baseurl, sid);
                doc = executeRequest(url, sessionKey, null, "GET");
                Console.WriteLine("Results: \n" + doc.InnerXml);
            }
            else
            {
                Console.WriteLine("Search returned zero results");
            }
        }
    }
}

Re: Push Splunk events from .NET application

Josh — Mon, 23 Aug 2010 07:08:30 GMT

Hi yes that is correct, however this particular application is running on hundreds of client machines and collecting logfiles would not be the optimal solution, you could collect logs onto one server and Splunk the logs from ther, but having critical data come from the application itself would be great. Also this opens the door for both Java and .NET application becoming another form of scripted input 🙂

Re: Push Splunk events from .NET application

gkanapathy — Mon, 23 Aug 2010 08:05:51 GMT

Honestly, it's probably easier and it will be far more efficient and scalable to just send the events via a plain network port.

Re: Push Splunk events from .NET application

gkanapathy — Mon, 23 Aug 2010 08:17:56 GMT

This is probably a lot more overhead than you need. All Splunk needs is plain text, either pushed over a plain TCP or UDP port, or read from a file. The REST API is much more expensive on the indexer and will limit scalability (which may not matter if you have low volume). I've never heard of anyone using this, and I can't really imagine many cases where I'd recommend it over a simple plain text socket output.

Re: Push Splunk events from .NET application

Josh — Thu, 16 Sep 2010 01:40:44 GMT

Do you have an exaple of how this can be done in .NET?

Re: Push Splunk events from .NET application

mosmondor — Mon, 03 Dec 2012 12:38:34 GMT

If the socket is used, how is it authenticated?