Internal Mapreduce in Splunk

[Arun_N_007]

Hi,

I need to know how map functions and reduce functions are constructed using search string?
In one of the white paper i came across finding first reduce command (Which cannot be parallelized)...

How it happens in Splunk internally? Where the intermediate data will be shown?

What is pretimechart (Intermediate command for timechart??)????

• Arun

[aweitzman]

Is this what you're looking for?

https://www.splunk.com/content/dam/splunk2/pdfs/technical-briefs/splunk-and-mapreduce.pdf

[Arun_N_007]

@aweitzman,

Yes i went though this doc but here they mentioned something like unparallelizable command.

If i find unparallelizable command as second command in search with 100 commands then will all other 98 commands will be converted into reduce processes?

What is pretimechart (Intermediate command for timechart??)????

[aweitzman]

According to the document, "pretimechart" is a summarization command that can be run in a parallelizable way across all the data sets generated by "map" commands, so that when it comes time to "reduce," all the unnecessary data is already pruned.

As for your "what if" scenario, since most search-related commands don't fall into that bucket, you shouldn't have to worry about it very often. And if you do run into it, then there are usually creative ways to restructure your search so as to avoid it.

[Arun_N_007]

Thank you Aweitzman.

Could you please have a look into my below posts :). Its been 2 weeks i didnt got any answers yet :(...

http://answers.splunk.com/answers/239222/when-to-use-django-approach-in-splunk-apps.html

http://answers.splunk.com/answers/237073/is-dynamic-query-construction-possble-in-splunk-wi-1.html