Alerting

real time alret action performance iisue

shavitpren
Loves-to-Learn

Hi,

I want to create a real time alert of about 3000 Messages per secend.

I want to create action for each message to create an http alert to another system.

my problam is that when I tryed to do that i recived about 50-100 messages per second and i got a big delay.

what is the best way to handle this throughput?

can we use batch in teal time laert?

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @shavitpren,

some question to identify the issue:

at first, what infrastructure are you using (in termes of architecture and recommended hardware?

Because performaces mainly depend on the available CPUs and especially on disk performaces, have you at least 800 IOPS on your disks?

are you sure that the server you're using to ingest events is able to index events without delays (outside the search)?

Then, is it mandatory to have a real time search? could it be a search scheduled e.g. every minute?

this second option is better for performances.

Ciao.

Giuseppe

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...