Getting Data In

C# API Query TImeOut

michaudel
Explorer

Hello,
I edited the c# splunk API search example and made it into the method below. It basically creates a connection then does a search and puts all the results into a list of csv. Which i use later.

the problem i am having is if the search takes a while to run, the connection gets closed so when i to too read each line from the stream i get an error saying "The request was aborted".

is there a way to set the timeout value?

or can i create the search, wait for it to finish, then re-connect to pull down the results.

   public static List<string> SplunkSearch(string strQuery)
    {
        var cli = Command.Splunk("search");
        cli.AddRule("search", typeof(string), "search string");

        cli.Opts.Add("host", "host");
        cli.Opts.Add("port", "8089");
        cli.Opts.Add("scheme", "https");
        cli.Opts.Add("username", "uname");
        cli.Opts.Add("password", "pwd");
        cli.Opts.Add("search",strQuery);
        cli.Opts.Add("Timeout", "1000");

        var service = Service.Connect(cli.Opts);

        var jobs = service.GetJobs();
        var job = jobs.Create((string)cli.Opts["search"]);

        while (!job.IsDone)
        {
            Thread.Sleep(1000);
        }



        var outArgs = new Args
        {
            { "output_mode", "csv" },

            // Return all entries.
            { "count", "0" }
        };
        int intRun = 0;
        string columnnames = "";
        List<string> lstResults = new List<string>();
        using (var stream = job.Results(outArgs))
        {
            using (StreamReader reader = new StreamReader(stream))
            {
                while (!reader.EndOfStream)
                {
                    lstResults.Add(reader.ReadLine());
                }
            }
        }

        return lstResults;
    }
Tags (2)
1 Solution

ywu
Splunk Employee
Splunk Employee

In your code, at the time the stream is read, search has finished already. It is a separate httpwebrequest to get the result. It is essentially what you said -- "wait for it to finish, then re-connect to pull down the results."

Do you have to more detailed info to share?

What many lines are there in the search result?

View solution in original post

0 Karma

ywu
Splunk Employee
Splunk Employee
0 Karma

ywu
Splunk Employee
Splunk Employee

In your code, at the time the stream is read, search has finished already. It is a separate httpwebrequest to get the result. It is essentially what you said -- "wait for it to finish, then re-connect to pull down the results."

Do you have to more detailed info to share?

What many lines are there in the search result?

0 Karma

ywu
Splunk Employee
Splunk Employee

Chunking the result set should be a good approach. There's first class support for this in Splunk REST API. Refer to 'count' and 'offset' parameters of

GET search/jobs/{search_id}/results

on

http://docs.splunk.com/Documentation/Splunk/5.0.2/RESTAPI/RESTsearch#search.2Fjobs.2F.7Bsearch_id.7D...

Using C# SDK, you can supply the two parameters using Args object which is a dictionary of name and value pairs. Let me know if you have any questions on this.

Btw, I am interested in exactly where it fails in your case now. So please send me any other info about the error if any.

0 Karma

michaudel
Explorer

Thank you for the explanation on the http request part. I think you are on to something, i think it is the size of result that is getting me. When i run a long query, but bucket into hour of day (so 24 results) it works even though the query took 5 mins. If I run another query which returns about 10K results,it aborts the connection. I am guessing i need to chunk my result set. Ask for 1K at a time or something?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...