Direct csv lookup vs. custom search command

[beaumaris]

We currently have an in-line csv table lookup that is used in both summary and normal index searches. Due to the needs of several customers, we would also like to specify a python script from the search command line (instead of the lookup command) that performs some direct results substitutions. We would like to keep the same syntax in the search command but be able to perform either type of lookup by simply replacing a .conf file or python script. Note that using macros is not appealing since we already have a sizable macros.conf and don't want to manage it per customer.

Customer A custom command: | providertitlelookup |

Customer B direct lookup: | lookup csv_provider_title Filename as Asset OUTPUT Provider, Title |

Ideally if we could keep the syntax of Customer A then we would just need a script to do the basic substitutions for customer A and a script to do the csv table lookups for customer B. We would use the syntax of Customer A in the baseline search and merely install the desired python script for each customer.

For the python script that supports Customer B, we will have to do the 'lookup' search directly from the python script. This search-within-a-search
is likely to be very slow compared to doing the lookup in the primary search. This seems like a lot of overhead to set up the context for each search. There are
potentially thousands of entries in the results set. An alternative might be to use Python csv to directly open the lookup table.

What is the recommended way to implement a csv table lookup within a python script that is called to process results sets? Is there another way to keep the same basic search-command syntax but perform one of (custom command, external lookup, db-lookup, macro) or other method to do either a lookup or result substitution?

[hazekamp]

beaumaris,

There are a couple of ways to approach this, but I might recommend using a custom python lookup for customers that need specific functionality. This would give you the ability to override transforms.conf for customers that need the python lookup, but keep the search syntax the same.

For instance:

Customer A:
## transforms.conf
[csv_provider_title]
external_cmd = csv_provider_title.py
external_type = python
fields_list = Filename, Provider, Title

Customer B:
## transforms.conf
[csv_provider_title]
filename = csv_provider_title.csv

Customer A/B Search:
<your search> | lookup csv_provider_title Filename as Asset OUTPUT Provider, Title