Alright I figured you would want the fields extracted with their intended fieldnames instead of any-and-all matches being contained in a single multivalue field so here is SPL to do that. <base_search>
``` this SPL required a field named "data" containing a raw string as its value ```
``` this can be macroed by replacing the input field "data" and lookup name "test_regex_lookup.csv" ```
``` example: | `extract_regex_from_lookup(data, test_regex_lookup.csv)` ```
``` pull in all regex patterns as an array of json objects into the parent search as a new field ```
| join type=left
[ | inputlookup test_regex_lookup.csv | tojson str(pattern_type) str(regex) output_field=regex_json | stats values(regex_json) as regex_json | eval regex_array=mv_to_json_array(regex_json) | fields + regex_array ]
``` parse array of json objects into a multivalued field of json objects ```
| eval
regex_json=json_array_to_mv(regex_array)
``` remove array (no longer needed) ```
| fields - regex_array
``` search the raw text of field "data" for matches against any of the regex patterns contained in the regex_json multivalue field ```
| eval
regex_match_json=case(
mvcount(regex_json)==1,
if(match(data, spath(regex_json, "regex")), json_set(regex_json, "matches", replace(data, ".*(".spath(regex_json, "regex").").*", "\1")), null()),
mvcount(regex_json)>1, mvmap(regex_json, if(match(data, spath(regex_json, "regex")), json_set(regex_json, "matches", replace(data, ".*(".spath(regex_json, "regex").").*", "\1")), null()))
)
``` remove regex_json (no longer needed) ```
| fields - regex_json
``` (optional) multivalued field containing all pattern matches ```
| eval
all_regex_matches=mvmap(regex_match_json, spath(regex_match_json, "matches"))
``` create temporary json object to hold key/value pairs for pattern_type attribution ```
| eval
tmp_json=json_object()
``` loop through the regex_match_json multivalue field and assign a key/value entry to "tmp_json" for the (pattern_type: matches) ```
| foreach mode=multivalue regex_match_json
[
| eval
tmp_json=json_set(tmp_json, spath('<<ITEM>>', "pattern_type"), spath('<<ITEM>>', "matches"))
]
``` full spath against tmp_json to get field extractions for all matches against the pattern_types ```
| spath input=tmp_json
``` remove temporary json object (no loger needed) ```
| fields - tmp_json
``` (optional) remove regex_match_json field ```
| fields - regex_match_json
``` end of `extract_regex_from_lookup(2)` macro ```
``` table all extracted fields derived from "data" field and regex stored in lookup "test_regex_lookup.csv" ```
| table _time, data, all_regex_matches, * I am pretty happy with how this turned out but there may be an easier way of doing it. Would be glad to hear anybody else to chime in on an easier way of accomplishing this. I have just always had problems with piping in data from a lookup into a parent search as executable SPL other than pulling it into an eval of some sort. Reference screenshot of sample output So the current SPL will assign the match to its corresponding row's pattern_type value from the lookup as a fieldname. In this example it is SSN, date, and name.
... View more