topic Re: How is Splunk utilizing Map Reduce? in Splunk Search

How is Splunk utilizing Map Reduce?

ashishebansal — Tue, 29 Jan 2019 13:09:54 GMT

How is Splunk utilizing Map Reduce and also, does it use the same tech for SPL and data compression?

Re: How is Splunk utilizing Map Reduce?

dkeck — Tue, 29 Jan 2019 13:24:22 GMT

HI,

maybe this could answer your question?

https://www.splunk.com/web_assets/pdfs/secure/Splunk_and_MapReduce.pdf

Re: How is Splunk utilizing Map Reduce?

ashishebansal — Tue, 29 Jan 2019 13:29:28 GMT

I already tried going through that and the other whitepaper "Technical Paper: Large-Scale, Unstructured Data Retrieval and Analysis Using Splunk".

However, I am still confused on the overall utilization of Map Reduce in Splunk.

Re: How is Splunk utilizing Map Reduce?

richgalloway — Tue, 29 Jan 2019 20:03:25 GMT

I think perhaps you're putting too much into the "Map Reduce" term. It probably doesn't mean the same thing as other places where it you've seen it used. In the Splunk world, it means search queries are sent to all indexers for processing. The indexers send their results back to the originating search head which combines the results and finishes processing the query.

Re: How is Splunk utilizing Map Reduce?

ddrillic — Tue, 29 Jan 2019 20:25:37 GMT

Very interesting @dkeck. The document shows the usage of the MapReduce concept, such as the following at the buckets level -