Add fields to events returned by inputlookup data with other data from the same inputlookup.

[livesplunkcomsk]

I have several lookup tables containing various data types filenames hashes emails usernames etc (lookup tables are separated by data type), each of these lookup tables also have a UUID column for a specific entry, so the CSV headers for filename date look like :

"fileName","uuid"

"fileName" data may actually only be a partial filename
Within the context of the CSV neither of these columns' data is unique, but together fileName+UUID data are.

QUESTION:
Given a query such as the one below, which returns interesting events, I need help implementing SPL to add a dict (for example: {"matchedValue": value, "UUIDS:[uuid1,uuid2,uuid**n]}) to each event, what SPL do I need to add?
-note this does not neccessarily need to be a dict, adding two fields to each event one "matchedValue" field and a "UUIDS" field with a delimited string of UUIDS works too.

index=USB_activity_data [|inputlookup interesting-filenames.csv | fields fileName | rename fileName as query]

END GOAL:
My goal is to push these modified events to another inhouse non-Splunk application, to achive this I've started working on my first Splunk App with the Python SDK (I've played with other Splunk Python apps before, but this is my first from scratch). I've framed a StreamingCommand in this app to format the event so our inhouse application can accept it and have another command that will do the posting.

[to4kawa]

UPDATED:

index=USB_activity_data 
[| inputlookup interestingnames.csv 
 | fields fileName
 | rename fileName as query] 
| stats values(FileName) as fileNames by USBDeviceID username
| mvexpand fileNames
| eval fileName=mvindex(split(fileNames,"/"),-1)
| inputlookup append=t interestingnames.csv
| stats delim="|" values(uuid) as uuid  values(username) as username by fileName 
| where isnotnull(username)
| nomv uuid
| rename fileName as matchedValue
| table matchedValue uuid

thanks for your sample log and the details.
maybe works.

---

create csv with match type: wildcard

A comma and space-delimited list of <match_type>(<field_name>) specification to allow for non-exact matching. The available values for non-exact matching are WILDCARD and CIDR. Specify the fields that use WILDCARD or CIDR in this list.

cf. lookup
your query:

index=USB_activity_data
| lookup interesting-filenames fileName as yourField OUTPUT fileName uuid
| append [|inputlookup interesting-filenames.csv]
| stats values(uuid) as uuid by fileName

[livesplunkcomsk]

So I may have been asking the wrong question, but I found a solution that works for me I needed WILDCARD(keyword) under the lookup definition Advanced settings and fileName data needed "" *on both sides.
Where "keyword" is the column name containing fileName data

index=USB_activity_data  [|inputlookup interesting-filenames.csv| fields keyword| rename keyword as FileName] |rename FileName as keyword |lookup interesting-filenames.csv keyword OUTPUT keyword as matchedValue uuid

I can then pipe this query into a table to pull all the fields I need.

[livesplunkcomsk]

No vote here.

The query you suggest does not work. It does not filter or appear to append fileName or UUID.
Your suggested query returns a sorted version of the below query containing only fileName and UUID columns

|inputlookup interesting-filenames.csv

Your suggestion returns ~177,000 events
WHEREAS the below query returns ~7700 matched events (FileName, USBDeviceID and username are fields extracted from the original events and independent of the inputlookup ), but I don't know how to properly map/append the matched fileName and UUID to the filtered events.

index=USB_activity_data 
    [| inputlookup interestingnames.csv 
     | fields fileName
     | rename fileName as query] 
|stats values(FileName) by USBDeviceID username

[livesplunkcomsk]

Given this sample event:

<36>Feb  7 00:00:15 DetectionProductHostname DetectionProduct: Unique-endpoint-Hostname,10.10.10.128,Continue, - Caller MD5=9ffffffffffffffff3333333eeeeeb,File Delete,Begin: 2020-02-07 00:00:21,End: 2020-02-07 00:00:21,Rule: Log files written to USB drives | Log writing to USB drives,54321,c:/path/to/somefile.exe,0,No Module Name,F:/path/to/another/file.TMP,User: unique-username,Domain: LOCAL,Action Type: ,File size (bytes): 0,Device ID: USBSTOR\Disk&Ven_Kingston&Prod_SNA-DC/U&Rev_1.14\f0ffffffffffffffff0f0f0f0f0f0f0&0

My use of the stats command would output:

USBDeviceID|username|FileName

USBDeviceID=USBSTOR\Disk&Ven_Kingston&Prod_SNA-DC/U&Rev_1.14\f0ffffffffffffffff0f0f0f0f0f0f0&0
username=unique-username
FileName=F:/path/to/another/file.TMP

So given every event match if fileName from the lookuptable is in the extracted field FileName, I want to append to each matched event, the value of fileName as "matchedValue" and UUIDS as a delimited list of UUID that submitted a value for fileName

[to4kawa]

@livesplunkcomskao2
thanks for the details. my answer is updated. please confirm.

[to4kawa]

index=USB_activity_data

Is there the field fileName?

[livesplunkcomsk]

there are filename fields, I have multiple indexes which have different names to represent file names. Some filename fields may actually be full paths to the file while others may be only the file name.

I know how to write the base queries to return event results I'm looking for.

What I don't know how to do is once those event results are returned is to add the fileName (which may be a partial of the filename value in the returned event) and UUIID.

A UUID is the designation for a submitter in the inhouse app.

So if two submitters submit the same string for fileName I want the "matchedValue" to be the string by the submitters and "UUIDS" to be a list of those two submitters.

[to4kawa]

index=USB_activity_data

Could you tell me this results and the value of fileName at that time?

sample:

| makeresults
| eval fileName="/opt/test/A#/opt/test/B#/opt/test/C"
| makemv delim="#" fileName
| mvexpand fileName
| rex field=fileName "^(?<filepath>.*(?<=\/))(?<fileName>.*)$"
| join type=outer fileName [ `comment("This simulates inputlookup.")`
|makeresults
| eval fileName=split("A#C","#")
| mvexpand fileName
| eval UUID=random()]

like above, maybe we can create the query.

[livesplunkcomsk]

Could you tell me this results and the value of fileName at that time?

I'm not sure I understand could you elaborate?