Looks like this question might have answered it using:
1. Hunk preprocessors (RecordReaders)
2. Streaming libraries
Looks like this question might have answered it using:
1. Hunk preprocessors (RecordReaders)
2. Streaming libraries