Splunk Search

Internal Mapreduce in Splunk

Arun_N_007
Communicator

Hi,

I need to know how map functions and reduce functions are constructed using search string?
In one of the white paper i came across finding first reduce command (Which cannot be parallelized)...

How it happens in Splunk internally? Where the intermediate data will be shown?

What is pretimechart (Intermediate command for timechart??)????

  • Arun
Tags (2)
0 Karma

aweitzman
Motivator
0 Karma

Arun_N_007
Communicator

@aweitzman,

Yes i went though this doc but here they mentioned something like unparallelizable command.

If i find unparallelizable command as second command in search with 100 commands then will all other 98 commands will be converted into reduce processes?

What is pretimechart (Intermediate command for timechart??)????

0 Karma

aweitzman
Motivator

According to the document, "pretimechart" is a summarization command that can be run in a parallelizable way across all the data sets generated by "map" commands, so that when it comes time to "reduce," all the unnecessary data is already pruned.

As for your "what if" scenario, since most search-related commands don't fall into that bucket, you shouldn't have to worry about it very often. And if you do run into it, then there are usually creative ways to restructure your search so as to avoid it.

Arun_N_007
Communicator

Thank you Aweitzman.

Could you please have a look into my below posts :). Its been 2 weeks i didnt got any answers yet :(...

http://answers.splunk.com/answers/239222/when-to-use-django-approach-in-splunk-apps.html

http://answers.splunk.com/answers/237073/is-dynamic-query-construction-possble-in-splunk-wi-1.html

0 Karma
Get Updates on the Splunk Community!

Traffic logs from Splunk Add-on for Cisco Meraki

Recently deployed this add-on, but it doesn't seem to bring back Traffic or URL logs like we did when using ...

Clear text password in command line

mogod command line argument having clear text password like "--sslPEMKeyPassword=password"how to avoid clear ...

Checksum mismatch on Deployment Clients which were rebuilt with same IP but different ...

We recently rebuilt a server which had splunk UF installed. After the rebuild, the IP remained same but ...