Splunk Search

Symphony Grid Tasks query hangs up while using Splunk REST API

lasnab82
Observer

Hi Splunk Admins, Hi Users,

I would like to give some background on our application. It is a C# application which runs as a console application in a distributed computing environment which uses IBM Platform Symphony grid as the hosting platform.

For better understanding of what goes on each grid node the application splunk logs are configured with the help of Splunk. The application takes one request at a time and does not support asynchronous execution. Each request is named as RunId in our world and splunk assigns one session to each request distinctively which can then be queried using that RunID or SessionID. Splunk can be queried on RunID or SessionID both.

We normally use Chrome browser to query the logs on splunk but recently to analyse the huge amount of data which is written in the logs we came up with a requirement to fetch the splunk query results via Splunk REST Api and use the results for analytical purposes like how much time did a Run take, memory consumed, Errors, crtitical data points etc etc.

While doing so the usual "Simple Query" as below works fine via the REST Api and gives us the desired result since its giving the RealTime output. But when we try to query the grid session query i.e. "Grid Query" it just gets hangs on the "GetSearchPreviewAsync" call.

Moreover, when we try to put a debug on the "GetSearchPreviewAsync" call and wait for a minute on breakpoint and then continue the execution then we do get some results if they are less in number. The same behaviour does not work if the results of the query is very big in number and it takes some time to execute the query.

Could you please advise what wrong we are doing here, need to make any changes in the code.

 

Screenshot 2020-11-03 102744.png

lasnab82_0-1604392028206.png

Thanks & Regards

Shobhit Bansal

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...