How to split events from the same log into different indexes based on content?

[himynamesdave]

I have logs that contains different customer IDs. I am intending to split different events from this log into different Splunk indexes based on the customer ID.

There are hundreds of customer IDs, with new ones added frequently.

I'm thinking of creating a lookup with customer ID and index to achieve this. I know I can use props and transforms to route events based on a regex, but is this possible to include a lookup here too?

It would also be good if I could set a default index should the lookup fail.

Can anyone give any advice on how they have (or would) approach and execute something like this.

[Jason · Accepted solution]

You can't do lookups at index time - they are only at search time.

You could probably, with some tweaking, index into a temporary index, then use a search with a lookup and the collect command, to re-index into new indexes.

You could certainly set up a big ORed regex to sort data out in Splunk... but it is probably easier to split the data somehow before indexing via some other tool/script.