Help on Correlation search scheduling

[ajmach343]

Hello,

I am currently building correlation searches in ES and I am running into a "searches delayed" issue. some of my searches run every hour, most are every 2 hours, and some every 3, 12 hours.

My time range looks like:

Earliest Time: -2h 
Latest Time: now

cron schedule: 1 */2 * * *

for each new search I add +1 to the minute tab of the cron schedule up to 59 and then start over. 
so on the next search the schedule would be 2 */2 * * * and so on...

is there a more efficient way I should be scheduling searches?

Thank you.

[marnall]

No that should be fine. As long as you have enough CPU and threads and your correlation searches are not overlapping with its next execution (e.g. if the search runs every 2 hours but it takes 2.5 hours to complete), then you use the +1 minute technique to spread the searches around, then it should be fine.

Do you get warnings about concurrent searches or do you see high CPU usage in your monitoring console?

[ajmach343]

I have about 800 searches. some that run take more than a minute. 

so in the messages it states: status: skipped, reason: "The maximum number of concurrent auto-summarization searches on this instance has been reached. " 

no warnings or errors. all messages have "INFO" right after date/time

cpu usage is at about 12% and memory usage is at 28%