I used a similar tool called JMeter with Splunk.. JMeter generated traffic which wrote logs to the server. There was a Splunk forwarder installed on the remote server which sent its logs to Splunk
Perhaps I mis-understood your question.. I used JMeter to load test an external application on a server and sent the results to Splunk to create a "real-time" dashboard to view performance during the load test. Are you referring to load testing Splunk itself? If so, why?
yes, i want to run load test on splunk
i want to demonstrate a situation that multiple users will use the system and run heavy queries and see how it affect on the server (cpu, ram, memory) and what is the performance of the query
Splunk scales horizontally, so you in theory could do it on a small scale and do the math to see how it would preform on a larger scale. A good rule of thumb is each concurrent search will consume 1 CPU core on the search head. You should also consider IOPS and disk type on indexers for your load test. Slow read times on disk will significantly slow down your testing. IMO, you're wasting your time load testing your Splunk deployment
You do not need to integrate any external tool to evaluate the performance of your Splunk. As splunk itself is a log monitoring tool and can very well deduce the external as well its internal logs to give you the performance metrics that you want.
At a high level, you can refer to the following Dashboards on Monitoring Console -
Overview (Check the CPU and Memory of all of your Splunk components of your distributed env)
Instances (Check the OS, CPU usage/process, RAM usage/process, Disk usage, Historical charts of these resource usage, Availability)
Search Activity (Adhoc + Scheduled searches, Resource usage specifically by search activities, search concurrency, Top memory consuming searches, Searches started per minute, Aggregate search Runtimes etc)
Resource usage (CPU cores, Avg load, % usage of memory, CPU and Disk Per Instance, machine or deployment, historical charts, I/O bandwidth utilization)
There are more dashboards that you might fit for your relevance.
Please accept this as answer if this resolves your query.