Splunk Search

Internal Mapreduce in Splunk

Arun_N_007
Communicator

Hi,

I need to know how map functions and reduce functions are constructed using search string?
In one of the white paper i came across finding first reduce command (Which cannot be parallelized)...

How it happens in Splunk internally? Where the intermediate data will be shown?

What is pretimechart (Intermediate command for timechart??)????

  • Arun
Tags (2)
0 Karma

aweitzman
Motivator
0 Karma

Arun_N_007
Communicator

@aweitzman,

Yes i went though this doc but here they mentioned something like unparallelizable command.

If i find unparallelizable command as second command in search with 100 commands then will all other 98 commands will be converted into reduce processes?

What is pretimechart (Intermediate command for timechart??)????

0 Karma

aweitzman
Motivator

According to the document, "pretimechart" is a summarization command that can be run in a parallelizable way across all the data sets generated by "map" commands, so that when it comes time to "reduce," all the unnecessary data is already pruned.

As for your "what if" scenario, since most search-related commands don't fall into that bucket, you shouldn't have to worry about it very often. And if you do run into it, then there are usually creative ways to restructure your search so as to avoid it.

Arun_N_007
Communicator

Thank you Aweitzman.

Could you please have a look into my below posts :). Its been 2 weeks i didnt got any answers yet :(...

http://answers.splunk.com/answers/239222/when-to-use-django-approach-in-splunk-apps.html

http://answers.splunk.com/answers/237073/is-dynamic-query-construction-possble-in-splunk-wi-1.html

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...