topic Re: Does the union command affect CPU utilization? in Monitoring Splunk

Does the union command affect CPU utilization?

kind7776 — Mon, 30 Apr 2018 06:24:28 GMT

Hi,

[architecture]
One search header, several indexers, one LB forwarder

[Question]
* If one search statement is returned, the search starts from one indexer. (Using CPU 1 core)

When using the Union command in the search header, does the search run in one indexer? (Use CPU 1 core?)
If not, does one search statement run on multiple indexers? (Using multiple CPUs?)
The point is, when using the Union command, does one search statement run on multiple indexers?

Thanks.

Re: Does the union command affect CPU utilization?

martin_mueller — Mon, 30 Apr 2018 10:08:25 GMT

Do post your search to get a more detailed answer.

In general, the streaming portion of searches (e.g. index=foo | eval field = "bar") will run on all indexers in parallel.
The same holds true for union'd searches, e.g. | union [search index=a | eval type = "foo"] [search index=b | eval mytype = "bar"] - which is the first example from the union docs at http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/union
Every indexer will run the searches in parallel, and return results to the search head.

For most cases, I'd recommend using OR instead of union: index=foo OR index=bar | ... because you also get parallel execution on all indexers for the streaming part but don't run into limits of the union command.

Re: Does the union command affect CPU utilization?

kind7776 — Thu, 03 May 2018 01:31:03 GMT

Sorry, I seem to have confused the question.
For example, using the append command, you can physically query one CPU core (one indexer)
If you have multiple indexers, I wonder if you use the union command to physically search the CPU cour using several indexers (multiple indexers).

I understand that append uses one cpu core, and union uses multiple cpu cores, so it is faster when using the union command.

I wonder if the above is true.

Re: Does the union command affect CPU utilization?

martin_mueller — Thu, 10 May 2018 19:08:00 GMT

If you have spare cores, consider enabling batch mode search parallelization: http://docs.splunk.com/Documentation/Splunk/7.1.0/Capacity/Parallelization#Batch_mode_search_parallelization

That will allow all batch mode eligible searches to search multiple non-hot buckets at once.

As for append vs union, I'd use neither in most cases - instead OR your data sets together in one big search.