But I'm struggling to figure out what you are even doing. It's like you are smashing several csv files together, sideways, doing some filtering on each then turning the whole thing on its head to make it look like it should have looked in the first place.
Optimizing this sort of thing should start at the beginning.
What is it you are trying to accomplish? What does the data look like? What should the output tell you?
Also, why is there no indexed data involved here at all? It feels like the better solution would be to index one or both "sets" of this data, and ... well, it'll work better and faster. But, I'll wait for the answers to the other questions first. 🙂