This is sorta opened ended. Since I am not too familiar with Real time searches short of just running a quick search.
I have about 40 users, who will on and off want to use a dashboard which is using 3 real time searches. Once more than 4-5 users are using Splunk sorta grinds to a halt. How can I get them to share the same output, rather than running their searches separately?
Any other best practices I should be aware of?
1) Resource estimating
2) Setting time limits?
3) Real time searches and searches/per cpu impact?
1 and 3 are the same. Each real-time search consumes 1 CPU core. You can add them as saved searches, and call the saved searches using the tags in your dashboard, rather than an in-line search. That should solve the problem you described, where multiple instances of the dashboard are consuming all of the CPU.
Honestly, best practice is to not use real-time. If you can schedule the searches to run on 1 minute intervals, it's far better utilization of resources.