Hello, My installation is the following: a cluster of three search head and three indexer all set on different server with large amount of RAM, CPU and disk available all running on a linux environment. When sending lots of logs (500 lines per seconds) to one of the indexers for prolongated amount of time I noticed something strange with my search heads; they start consuming huge amounts of CPU at seemingly random moments to the point where systemd takes several seconds to responds and end up failing some services on the server. The rest of the time, everything runs smoothly and the logs are ingested normally without any kind of problems. I noticed this problem because a monitoring service received a DBus connection timeout (5s timeout parameter) which seems to be quite extreme given the available resources on this server. Interestingly enough, the problem is only raised at precise timestamps happening every 15 minutes (XXh00, XXh15, XXh30 or XXh45) but I have no correlation rules or other user defined rules with such a timing. Any idea of what could cause such an excessive resource usage? Thanks
... View more