Handling Data with multiple formats

[jhallman]

Has anyone worked with parsing multiple formats within a log

Example we logs like driver.log for our Datasynapse Grid processing
and at least 5 different distinct formats

mutil-line format

[LOG|DEBUG|2011 August 23, 08:25:27 (622)|MEMORY_DEBUG|ResponseCallbacks-1: DriverJobSpace$1|162.103.129.63 (wppsa01a0038.wellsfargo.com)]
In FuBaseWebProcJob::processTaskOuput(0) - heap size(50,577,408) free(8,822,904) % free(17.444357765427597)
[END]

2nd mutli-line
Bond has been loaded from Calypso
putting bond into cache cusip 3133XYJ97
SourceHit=22.0 CacheHit=5.0 HitRate=18.519
**** out of sync block
*********BondSettleDays =1 tradeSd=08/24/2011
::grName::gridlib_smiley2_prod_ro
Resetting DATASYNAPSE_RETRIES to 0
[2011-08-23 08:25:25.805] CARE Domain: MSRBTaskTimeoutMin=null
[2011-08-23 08:25:25.805] Executing grid job...

And at least 3 single line formats

08/23/11 08:25:27.627 INFO: [ServiceEvent] CompletedTask:TradeAnalyticsJob:3133XYJ97-8293306600710979712-0:Total:1

CARESERVICE END:CE0C1AE5-E762-4474-9541-E8724CFD8C86|45|S|3133XYJ97: TIME::8/23/11 11:59:00.674 PM EDT

CalypsoServiceGrid Response has been posted. 27.0#27.0

[woodcock]

I assume the problem is that these variants are all inside of a single file. This blog does a good job of explaining how to handle that:

http://www.function1.com/2013/01/oh-no-splunking-log-files-with-multiple-formats-no-problem