topic Re: Field extraction on lookup table in Splunk Search

How to perform a field extraction on a field from a lookup table?

dkorlat — Wed, 05 Aug 2020 06:02:00 GMT

Hi,

How to perform a field extraction on a field from a lookup table?

I'm trying to add another field so the data model in Splunk Enterprise Security can recognise the field.

The issue i'm having is field extraction in props.conf and transforms.conf happen before the lookup table.

I tried the AS command after OUTPUT on the lookup, but it renames the default field from the Windows Add-on. I only want to add another field and not rename the fields in the Add-on. REPORT- in props.conf and transforms.conf works on any other field except fields from lookup tables.

I need to perform the field extraction in the Add-on and not in SPL.

Thanks in advanced.

Re: Field extraction on lookup table

isoutamo — Wed, 29 Jul 2020 11:45:38 GMT

here is describing the sequence of search-time operations https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Searchtimeoperationssequence. It shows that lookups are applied after transforms. For that reason I think that the only way you can so it is SPL not props.conf or transforms.conf.

r. Ismo

Re: Field extraction on lookup table

dkorlat — Thu, 30 Jul 2020 08:44:44 GMT

Thanks,

Splunk Enterprise Security requires the field for the CIM to build the data model.

I won't be able to run it as a SPL as the data models are built as a background task.

Re: Field extraction on lookup table

isoutamo — Thu, 30 Jul 2020 13:18:08 GMT

can you do that extraction before you are creating this inputlookup table (e.g. just add additional column there)?

r. Ismo

Re: Field extraction on lookup table

dkorlat — Fri, 31 Jul 2020 03:02:34 GMT

I need to create another field from the field generated by the table lookup.

Here is the line which creates the lookup table field

"LOOKUP-privilege_for_windows_security = windows_privilege_lookup privilege_id OUTPUT privilege"

I can use LOOKUP-privilege_for_windows_security = windows_privilege_lookup privilege_id OUTPUT privilege AS MyNewField with works, but I lose the field name privilege, which might cause other dashboards to stop working.

I can't post the props.conf as it exceeds 20000 characters.

Re: Field extraction on lookup table

shivanshu1593 — Tue, 04 Aug 2020 22:23:51 GMT

I got your requirement now, here's what you can try:

1. In the Index field in your datamodel, append the results of your lookup (inputlookup append=t your_lookup.csv)

2. In the calculated fields, use the option of extract more fields, and use Auto extracted fields and check if you can find your desired field there, if yes, just add it to your datamodel.

3. If you cannot find it via Auto extract, you can always go for the trusted Regular Expressions.

Try this and let me know if it works.

If it helps, please accept it as an answer.