Splunk Search

Internal Mapreduce in Splunk

Arun_N_007
Communicator

Hi,

I need to know how map functions and reduce functions are constructed using search string?
In one of the white paper i came across finding first reduce command (Which cannot be parallelized)...

How it happens in Splunk internally? Where the intermediate data will be shown?

What is pretimechart (Intermediate command for timechart??)????

  • Arun
Tags (2)
0 Karma

aweitzman
Motivator
0 Karma

Arun_N_007
Communicator

@aweitzman,

Yes i went though this doc but here they mentioned something like unparallelizable command.

If i find unparallelizable command as second command in search with 100 commands then will all other 98 commands will be converted into reduce processes?

What is pretimechart (Intermediate command for timechart??)????

0 Karma

aweitzman
Motivator

According to the document, "pretimechart" is a summarization command that can be run in a parallelizable way across all the data sets generated by "map" commands, so that when it comes time to "reduce," all the unnecessary data is already pruned.

As for your "what if" scenario, since most search-related commands don't fall into that bucket, you shouldn't have to worry about it very often. And if you do run into it, then there are usually creative ways to restructure your search so as to avoid it.

Arun_N_007
Communicator

Thank you Aweitzman.

Could you please have a look into my below posts :). Its been 2 weeks i didnt got any answers yet :(...

http://answers.splunk.com/answers/239222/when-to-use-django-approach-in-splunk-apps.html

http://answers.splunk.com/answers/237073/is-dynamic-query-construction-possble-in-splunk-wi-1.html

0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with Brett Adams

In our third Spotlight feature, we're excited to shine a light on Brett—a Splunk consultant, innovative ...

Index This | What can you do to make 55,555 equal 500?

April 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Say goodbye to manually analyzing phishing and malware threats with Splunk Attack ...

In today’s evolving threat landscape, we understand you’re constantly bombarded with phishing and malware ...