Getting Data In

C# API Query TImeOut

michaudel
Explorer

Hello,
I edited the c# splunk API search example and made it into the method below. It basically creates a connection then does a search and puts all the results into a list of csv. Which i use later.

the problem i am having is if the search takes a while to run, the connection gets closed so when i to too read each line from the stream i get an error saying "The request was aborted".

is there a way to set the timeout value?

or can i create the search, wait for it to finish, then re-connect to pull down the results.

   public static List<string> SplunkSearch(string strQuery)
    {
        var cli = Command.Splunk("search");
        cli.AddRule("search", typeof(string), "search string");

        cli.Opts.Add("host", "host");
        cli.Opts.Add("port", "8089");
        cli.Opts.Add("scheme", "https");
        cli.Opts.Add("username", "uname");
        cli.Opts.Add("password", "pwd");
        cli.Opts.Add("search",strQuery);
        cli.Opts.Add("Timeout", "1000");

        var service = Service.Connect(cli.Opts);

        var jobs = service.GetJobs();
        var job = jobs.Create((string)cli.Opts["search"]);

        while (!job.IsDone)
        {
            Thread.Sleep(1000);
        }



        var outArgs = new Args
        {
            { "output_mode", "csv" },

            // Return all entries.
            { "count", "0" }
        };
        int intRun = 0;
        string columnnames = "";
        List<string> lstResults = new List<string>();
        using (var stream = job.Results(outArgs))
        {
            using (StreamReader reader = new StreamReader(stream))
            {
                while (!reader.EndOfStream)
                {
                    lstResults.Add(reader.ReadLine());
                }
            }
        }

        return lstResults;
    }
Tags (2)
1 Solution

ywu
Splunk Employee
Splunk Employee

In your code, at the time the stream is read, search has finished already. It is a separate httpwebrequest to get the result. It is essentially what you said -- "wait for it to finish, then re-connect to pull down the results."

Do you have to more detailed info to share?

What many lines are there in the search result?

View solution in original post

0 Karma

ywu
Splunk Employee
Splunk Employee
0 Karma

ywu
Splunk Employee
Splunk Employee

In your code, at the time the stream is read, search has finished already. It is a separate httpwebrequest to get the result. It is essentially what you said -- "wait for it to finish, then re-connect to pull down the results."

Do you have to more detailed info to share?

What many lines are there in the search result?

0 Karma

ywu
Splunk Employee
Splunk Employee

Chunking the result set should be a good approach. There's first class support for this in Splunk REST API. Refer to 'count' and 'offset' parameters of

GET search/jobs/{search_id}/results

on

http://docs.splunk.com/Documentation/Splunk/5.0.2/RESTAPI/RESTsearch#search.2Fjobs.2F.7Bsearch_id.7D...

Using C# SDK, you can supply the two parameters using Args object which is a dictionary of name and value pairs. Let me know if you have any questions on this.

Btw, I am interested in exactly where it fails in your case now. So please send me any other info about the error if any.

0 Karma

michaudel
Explorer

Thank you for the explanation on the http request part. I think you are on to something, i think it is the size of result that is getting me. When i run a long query, but bucket into hour of day (so 24 results) it works even though the query took 5 mins. If I run another query which returns about 10K results,it aborts the connection. I am guessing i need to chunk my result set. Ask for 1K at a time or something?

0 Karma
Get Updates on the Splunk Community!

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Cultivate Your Career Growth with Fresh Splunk Training

Growth doesn’t just happen—it’s nurtured. Like tending a garden, developing your Splunk skills takes the right ...