Answers to these questions: What are bundles? When initiating a distributed search, the search head replicates and distributes its knowledge objects to its search peers. Knowledge objects include saved searches, event types, and other entities used in searching across indexes. The search head needs to distribute this material to its search peers so that they can properly execute queries on its behalf. The set of data that the search head distributes is called the knowledge bundle. The indexers use the search head's knowledge bundle to execute queries on its behalf. When executing a distributed search, the indexers are ignorant of any local knowledge objects. They have access only to the objects in the search head's knowledge bundle. The process of distributing knowledge bundles means that indexers by default receive nearly the entire contents of all the search head's apps. If an app contains large binaries that do not need to be shared with the indexers, you can reduce the size of the bundle by means of the [replicationWhitelist] or [replicationBlacklist] stanza in distsearch.conf. See "Limit knowledge bundle size" in this manual. The knowledge bundle gets distributed to the $SPLUNK_HOME/var/run/searchpeers/ directory on each search peer. Because the search head distributes its knowledge, search scripts should not hardcode paths to resources. The knowledge bundle will reside at a different location on the search peer's file system, so hardcoded paths will not work properly. By default, the search head replicates and distributes the knowledge bundle to each search peer. For greater efficiency, you can instead tell the search peers to mount the knowledge bundle's directory location, eliminating the need for bundle replication. When you mount a knowledge bundle, it's referred to as a mounted bundle. To learn how to mount bundles, read "Mount the knowledge bundle". What is bundle replication? Essentially search heads send the required knowledge objects so that the indexers/peers can run the searches on its behalf. You can turn it off if you have mounted bundles. What is the impact? Bundle replication happens asynchronously from search. There is some impact on performance. However, long bundle replication times mean that the changes made in the search head will take longer to become effective, since they become effective only after the indexers have the bundles. (this is straight from one of the devs) Mounting your knowledge bundles would help as detailed here http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Mounttheknowledgebundle However, mounting to any type of share over WAN is going to yield very poor results and is not recommended by splunk. And to clarify even further, a search will not be prevented from running just because knowledge replication has not finished. In the case that a search is launched from the search head mid bundle replication, the search head will tell the peers to use the last common bundle set, ie the last updated bundle they received. This prevents erroneous results and makes sure the peers use the same bundle for that search. ... View more

I'll consult with our devs, but you should file this as an enhancement request. ... View more

First off, thanks for bringing this up in the channel today. Excellent question! Contrary to our beliefs, the answer is YES. YAY. To configure this you'll have to reference server.conf http://docs.splunk.com/Documentation/Splunk/5.0/admin/Serverconf Straight from the docs: ########################################################################################################## # Set this node to be a searchhead to multiple cluster masters - # "SplunkMaster01" with pass4SymmKey set to 'someSecret and "SplunkMaster02" with no pass4SymKey set here. ########################################################################################################## [clustering] mode = searchhead master_uri = clustermaster:east, clustermaster:west [clustermaster:east] master_uri=https://SplunkMaster01.example.com:8089 pass4SymmKey=someSecret [clustermaster:west] master_uri=https://SplunkMaster02.example.com:8089 I would, however, caution you that, of course, if you have duplicate data in both datacenters, then you will end up with duplicates in your search. Simple workaround is to add dedup _raw to the base of your searches. Additionally, depending on where your search head resides, you might run into latency issues if one cluster takes significantly longer to stream results back to the search head than the other. ... View more

The easiest way to extract that field will be to use the rex command in search. This should give you what you want: <yoursearch> | rex field=_raw "order:\s(?<netsales>\d+.?\d+)" That should extract that dollar amount into a field called netsales. This is assuming that the dollar amount is always preceded by order: ... View more