topic Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries) in Getting Data In

Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Wed, 30 Sep 2020 04:12:48 GMT

Hey guys,
I have an online connection with another web service Serv_1:
A. it sends data to MySplunk via online REST API
B. I ran a search in MySplunk to enrich the data
C. I send the enriched data back to Serv_1

Question:
What if Serv_1 sends me an enormously many API requests to MySplunk?
Do I need smth a queue manager like TIBCO between MySplunk and Serv_1?
Or my Splunk search heads cluster will do just fine?

Thanks in advance.

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

nickhills — Sat, 15 Feb 2020 09:15:58 GMT

A.)
How are you sending data to Splunk - do you mean via the Http Event Collector (HEC)?
If so, the HEC is built to handle a large number of simultaneous requests, however you will need to monitor your deployment and may consider load balancing the incoming data across a 'pool' of HECs.

B.)
No Problem Here 🙂

C.)
There are a few ways to do this, but it probably depends on your exact use case.
Option 1.
Build an 'alert' which runs every x minutes to processes your data in Splunk and uses an alert action to send the data via a script back to your web service. (This is probably the approach I would take)

Option 2.
Run a saved search every x minutes which processes your data.
Create a process either on your webservice, or another host to use the Splunk Rest API and request the job results.
Transform your results as required and submit them to your webservice.

You should not need any other software (other than your custom alert/scripts) to process the data.
Whether your SHC "will do just fine" is dependent on your overall architecture and Splunk performance, but if your Splunk environment is in good health, I would think you should be ok.

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

jkat54 — Sat, 15 Feb 2020 11:19:34 GMT

HEC will give 500 errors when over worked. If your data is crucial to you, then yes I recommend sending data in via some method that can guarantee delivery. Such as tibco, apigee, etc. or write your own code that guarantees the delivery when any status code greater than 399 is returned.

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Sat, 15 Feb 2020 11:44:40 GMT

Hi thanks for the quick response! 🙂

I don't use HEC, and I need to reply to the Serv_1 as quickly as possible in the same API session (I guess...).

Saved searches by schedule not applicable in my use case...

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Sat, 15 Feb 2020 11:47:37 GMT

Hi, thank you for the quick response! 🙂

I don't use HEC, and I need to reply to the Serv_1 as quickly as possible in the same API session (I guess...)...
Seems that i am forced to use some sort of smart/web service to handle the situations whe I get large number of incoming calls to MySplunk search service...

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

jkat54 — Sat, 15 Feb 2020 12:53:51 GMT

That or maybe you could setup heavy forwarders to receive the data and then forward to splunk cloud.

I recommend talking to your account rep and asking for assistance.

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Sat, 15 Feb 2020 12:57:08 GMT

Hi thanks, but what is"rep" ? 🙂

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

jkat54 — Sun, 16 Feb 2020 04:32:05 GMT

Representative, apologies. Rep is short for representative.

Maybe you call them your account manager?

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Sun, 16 Feb 2020 11:14:40 GMT

Not my case, unfortunately 🙂
Thanks

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

jkat54 — Sun, 16 Feb 2020 12:32:32 GMT

I guess you can always call on a partner for consulting. It sounds like you could really use the help of someone with experience but would be best if you had them for a couple of days maybe a whole week. This might take days, weeks, or months to sort out here question by question.

You're always welcome to post as many questions here as you need though!

Cheers and hope we can help!

Re: Do I need TIBCO or smth like that (highload to Splunk REST API queries)

highsplunker — Sun, 26 Apr 2020 11:47:54 GMT

@nickhillscpl thanks a lot!