My data consists of pairs of files, lets call them file_A_1...file_A_n, and file_B_1...file_B_n, where file_A_1 is connected with file_B_1.
The pairs are always ingested at the same time together. The first step I need in my datamodel is to join the corresponding pairs, like
source=file_A_1 join type=outer myIDField [ search source=file_B_1 ]. How can I achieve this dynamically, with every pair of file?
If myIDField is unique across all files, do you really need to join the two files together?
source=file_A* OR source=file_B* | stats values(*) AS * by myIDField
If the myIDField value is not unique, you could also extract the id number from the file.
source=file_A* OR source=file_B* | rex field=source "_(?<file_id>\d+)" | stats values(*) AS * by myIDField, file_id
Here is a handy chart on when to use the various aggregators
http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation