Alerting

real time alret action performance iisue

shavitpren
Loves-to-Learn

Hi,

I want to create a real time alert of about 3000 Messages per secend.

I want to create action for each message to create an http alert to another system.

my problam is that when I tryed to do that i recived about 50-100 messages per second and i got a big delay.

what is the best way to handle this throughput?

can we use batch in teal time laert?

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @shavitpren,

some question to identify the issue:

at first, what infrastructure are you using (in termes of architecture and recommended hardware?

Because performaces mainly depend on the available CPUs and especially on disk performaces, have you at least 800 IOPS on your disks?

are you sure that the server you're using to ingest events is able to index events without delays (outside the search)?

Then, is it mandatory to have a real time search? could it be a search scheduled e.g. every minute?

this second option is better for performances.

Ciao.

Giuseppe

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.