Splunk Search

split based on lookup

coenvandijk
Observer

Hello,

I have a set of windows events (4656 and 4663) which contain fullpathnames. I also have a list of 'critical' directories as a lookup. I would like to split the fullpathname field into 2 new fields: One containing the 'critical' directory and one containing the remainder.

Example:
Full path: T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief\OPF\gedragscode\2007\brief aan A. Atmopawiro inzake gedragscode 2007.doc
Critical directory: T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief
Remainder: OPF\gedragscode\2007\brief aan A. Atmopawiro inzake gedragscode 2007.doc

What is the best option to achieve this?

best regards,
Coen van Dijk

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

Here's a run-anywhere sample that creates a rex.

| makeresults 
| eval critical="T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief!!!!C:\George\Washington\!!!!T:\k-share\Mickey.Mouse+Minnie*Mouse" 
| makemv delim="!!!!"  critical 
| table critical 
| mvexpand critical 
| rename COMMENT as "everything above this just generates test data with each critical directory in a single field called critical"

| rename COMMENT as "now we begin to format the records into a regular expression that will find directory1 or directory 2 etc"
| format "(?i)^(?<critical>" "" "3" "" "|" ")"
| rename search as searchRex
| rename COMMENT as "slashes have already been escaped, but we have to escape * and . and +"
| rex mode=sed field=searchRex "s/([\.\*\+])/!!!\1/g"
| rex mode=sed field=searchRex "s/!!!/\\\/g"
| rename COMMENT as "kill the field name and associated quotes"
| rex mode=sed field=searchRex "s/ critical=//g"
| rex mode=sed field=searchRex "s/\"  \| \"/|/g"
| rex mode=sed field=searchRex "s/> \"/>/g"
| rex mode=sed field=searchRex "s/\"  \)/)(?<remainder>.*)$/g"
| eval searchRex = "\"".searchRex."\""

The sample rex looks like this -

"(?i)^(?<critical>T:\\k-share\\A&O IR\\Bestuursrelaties\\W_BR_Archief|C:\\George\\Washington\\|T:\\k-share\\Mickey\.Mouse\+Minnie\*Mouse)(?<remainder>.*)$" 

Now, just in case that looks overcomplicated, let me get you a simple example of how it ends up -

| rex field=fulldirectory  "(?i)^(?<critical>T:\\critical\\directory1|T:\\critical\\directory2|T:\\critical\\directory3)(?<remainder>.*)$" 

That rex will put the value of any matching directory into the field named "critical" and then put anything left over into the field "remainder".

Unfortunately, the rex command does not natively take a variable for the regular expression string, so we're going to have to use the map command to have that happen.

And, map is a bit finnicky, so for testing you will need to use head to limit the results, and probably run a few times before you get it all to work right together.

 | map search="search  ...your other search terms here... | use head 5  here for testing | rex field=fulldirectory $searchRex$ | ...remainder of your search..."

View solution in original post

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Here's a run-anywhere sample that creates a rex.

| makeresults 
| eval critical="T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief!!!!C:\George\Washington\!!!!T:\k-share\Mickey.Mouse+Minnie*Mouse" 
| makemv delim="!!!!"  critical 
| table critical 
| mvexpand critical 
| rename COMMENT as "everything above this just generates test data with each critical directory in a single field called critical"

| rename COMMENT as "now we begin to format the records into a regular expression that will find directory1 or directory 2 etc"
| format "(?i)^(?<critical>" "" "3" "" "|" ")"
| rename search as searchRex
| rename COMMENT as "slashes have already been escaped, but we have to escape * and . and +"
| rex mode=sed field=searchRex "s/([\.\*\+])/!!!\1/g"
| rex mode=sed field=searchRex "s/!!!/\\\/g"
| rename COMMENT as "kill the field name and associated quotes"
| rex mode=sed field=searchRex "s/ critical=//g"
| rex mode=sed field=searchRex "s/\"  \| \"/|/g"
| rex mode=sed field=searchRex "s/> \"/>/g"
| rex mode=sed field=searchRex "s/\"  \)/)(?<remainder>.*)$/g"
| eval searchRex = "\"".searchRex."\""

The sample rex looks like this -

"(?i)^(?<critical>T:\\k-share\\A&O IR\\Bestuursrelaties\\W_BR_Archief|C:\\George\\Washington\\|T:\\k-share\\Mickey\.Mouse\+Minnie\*Mouse)(?<remainder>.*)$" 

Now, just in case that looks overcomplicated, let me get you a simple example of how it ends up -

| rex field=fulldirectory  "(?i)^(?<critical>T:\\critical\\directory1|T:\\critical\\directory2|T:\\critical\\directory3)(?<remainder>.*)$" 

That rex will put the value of any matching directory into the field named "critical" and then put anything left over into the field "remainder".

Unfortunately, the rex command does not natively take a variable for the regular expression string, so we're going to have to use the map command to have that happen.

And, map is a bit finnicky, so for testing you will need to use head to limit the results, and probably run a few times before you get it all to work right together.

 | map search="search  ...your other search terms here... | use head 5  here for testing | rex field=fulldirectory $searchRex$ | ...remainder of your search..."
0 Karma

adonio
Ultra Champion

you can create a field for the critical directory
here is an example:
https://answers.splunk.com/answers/468028/regex-source-and-destination-files-with-path-filen.html

0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...